Budget Committee - Regular Meeting

Call City Council Audit Committee to order. Roll call please. Council member Johnson. Here. Committee member Delworth. Committee member Donnelly. Here. Committee member Loesner. Here. Committee member McVeigh. Here. Chair Naker? Here. Welcome everyone to our March Audit Committee meeting and a happy St. Patrick's Day to all.

We have a robust agenda today so I want kick us off with introducing two new committee members who we are really excited to be welcoming to our group. One of them is chair of our housing redevelopment authority and my colleague on the city council council member Shaniqua Johnson. And another and our new community adviser is Dylan Donnelly. And I'm going to let them each take a minute to introduce themselves to the committee so we can get to know them. Starting with Chair Johnson.

Great. Well, it's a pleasure to be here. My name is Shaniqua. I'm the Ward seven City Council Member on the East Side Of Saint Paul, representing Dain's Bluff, Battle Creek, Conway, Eastview, Highwood Hills, Sweet Hollow, and Dayton's Bluff. I am really excited to just be able to join the audit committee. I think the work that we do here also sets the tone work for the policies that we make later. And so just excited to be here. Was thrilled to to be one of the council members selected to be on on the committee. And I guess I just had baby, so I'm coming up from maternity leave for three months. And so just getting re re back to being here fully and my son Isaiah is at home and I look forward to going to him after this.

But genuinely just excited to share space. I don't have a baby photo or I'd put it up on the monitor but if you want to see one I have plenty on my phone so I can show you after.

He can be an honorary visitor to the

Audit He may pop up every now and then. Honorary community advisor.

Honorary advisor, yes.

He has a

lot of advisors.

He might have plenty to say actually.

And actually, to that point about getting home, one of the things about the audit committee that I pride ourselves on is that we usually end on time, very respectful folks' time. Good committee to be on with work life balance. Mr. Donnelly, do you want to introduce yourself?

Dylan Donnelly, thank you for having me. My background is number one, St. Paul lifer. Background in manufacturing, workforce development, and I'm very enthusiastic about continuous improvement. So I'm looking forward to working with everybody here.

Great. Thank you And for being congratulations.

Thank you.

It's a competitive pool. I thought it might be nice for each of us to just say maybe a couple of things about ourselves just so our two new committee members know us. I think folks know me. So Mr. Lostetter, do you want to start? And then we can go maybe to Mr. McVeigh, Mr. Dilworth?

Sure. I'm Jeremy Lostetter. I live in Ward 1, my wife and two kids. And in my outside of this, I'm an internal audit manager at Hennepin County. I've been on the audit committee here for I forget now because I think it's coming up on my three years. Yeah. So like two and a half or something like that. And it's been great. It's been a great way to contribute to my city and meet a lot of great people and help shape the audit work that we do as part of this committee. So welcome new members.

Shaniqua Dillon, welcome to the committee. I'm really glad to have both of you here. I'm Noah McVay. I used to be the newest community adviser, but I'm not an agent. Happens fast. It does happen fast. I'm I'm I'm feeling so senior now. But I actually have a background in public administration and governmental accounting. I'm working my master's degrees, two different master's degrees in that. And by day, I actually am a substitute teacher for 12 schools in four different districts. Science, social studies, math mainly sixth grade through 12. So I get around. So I'll stop now.

And you'll learn this later this meeting, but Noah also chairs our community engagement committee of one which informs people about the audit committee. Our community engagement subcommittee, I should say. Ms. Doroth.

First of apologize for being late. No worries. I'm a director of budget and finance and facilities at McCollege of Design at the University of Minnesota. We are in budget season and I had to get some So documents in I was trying to meet a deadline. So but I have what, two kids, three grandkids and looking forward to retiring in August. So I'm on the last six months and it's just been quite hectic. Yeah. As we're trying to do that. But I've been on the audit committee. This is my the first of my second term. So enjoy being here and working for the city.

Thanks, Ms. Dulworth. Just a note to self that you can always re up. Yes. Great. Well, welcome again to our new members. Really excited to have you. And you're coming in at a great time. We as a committee, as you know, conduct performance audits. We typically have one audit going at a time.

And during that time we hear updates on the audit. We usually hear sort of a midpoint update and a final update before the audit goes to the city council for a presentation there. And then during that time we're also doing engagement to the community and trying to take ideas and think and talk about our own ideas for what the next audit topic will be. We have continuously striven to develop a calendar where we do the same thing every year and we're always on the same cycle. But for a number of reasons, we haven't quite been able to get into a perfect rhythm.

For one reason, recently we had a new mayor come in and an administration turnover. And as you saw last week, mister Donnelly, or last time we were here, we had a great conversation with the mayor and her team about what topics she might be interested in exploring. So rather than kind of charging ahead with our own topics, we wanted to wait and give her team a chance to weigh in. So one of the things we'll do today is debrief that conversation a little bit, see if people had anything come out from that that they were really interested in proposing as an audit topic. And another reason we sometimes get off of schedule is because our current audit might take longer than expected, and that's been the case here.

Doctor. Britt from Wilder Research will talk about this momentarily, but there have been some setbacks in that work for a lot of reasons that were out of Wilder's control. So we're getting sort of a later midpoint update than we normally would. And that is totally fine. So with that we will move into our second item on the agenda which is the data practices study update. And I'll welcome up Doctor. Heather Britt from Wilder Research who is our on retainer research firm and has done a number of wonderful audits for us so far. We're excited to hear what's going on. Welcome.

Great. Thank you.

She doesn't always dress like this.

I love it. I

think you're automatically on already. Okay. Yeah.

All right. Perfect. Right. Well, thanks everybody. Welcome. Excited to be here. And yes, appreciate your patience and grace as we have had a couple of tricky months in the Twin Cities. And that is part of what has shaped us being a little bit later in this process than we had, I think, hoped for and anticipated. So I'm gonna do a little bit of a reminder of where we are, what our key questions were. And then I'm just gonna give you an update on where we are in really our data collection process.

And then just give you a refresh on what we anticipate our timeline is going to look like. So just a broad reminder of what our overall purpose is related to this work. So effectiveness, efficiency, and compliance of the city's data practice request process. The tools, policies, and the user experience with really a focus on kind of internally the city's GovQA system, which is the internal request sort of tracking system, and its alignment with the state's data practices act requirements. So we've got kind of five sets of questions that as we went through our sort of first round of scoping, and then really kind of narrowed and sort of sieved and got clear about what that scoping looks like.

So I'm just going to review what those key question areas look like. So first one really focused on just GovQA, the capabilities, and the internal use of that system. So, what do those standard capabilities look like? How has the city configured it? How's that use documented?

And does it have sort of sufficient controls to really effectively manage what that process looks like internally? And that's not just for Nia and Greg, who are kind of what I call the super users for that system, but that's really each one of the department leads across the 16 departments that are really sort of significantly using GovQA. The second area that we're focused on, really taking a look at reporting and data trends. So what kind of reports are broadly available to the city staff? Which of those reports were just kind of out of the box reports that were available?

What were those kind of specifically created reports for the city? And what are those high level aggregated reports kind of rolled up? Tell about volumes, trends, timeliness, kind of the patterns. Since GovQA was really turned on for the purposes of the city, and it's tracking and management of these data requests coming through the city. The third area that you are really interested in is kind of that resident facing portal. So you all know that there's a portal that's available for folks to actually be able to put in requests. So what is that online portal? How do folks sort of use that portal? How does that function? What's the experience of folks?

And particularly, how do folks think about that in comparison to other systems that they might be using? So is there an opportunity for us to get some feedback about improvements related to that portal? The fourth area that you all were interested in, kind of that policies, training, compliance. What kind of training programs and guidance are offered to city staff? Are those adequate to ensure kind of compliance with state expectations?

And just kind of comparison related to really the data practice office expectations. Are there opportunities for any sort of improvements or sort of improved alignment there? And then finally, and this was a little bit of a pivot as we were doing really that sort of refinement in what the scope looked like. Is there an opportunity for comparison of what's going on in St. Paul to some other jurisdictions?

So, what are the patterns and practices used elsewhere? And this is where you all were really curious about what's going on in Hennepin County, Ramsey County, Minneapolis, Bloomington, and Duluth. What's the tool that they use internally for managing? What's that tool that they're using as it relates to the portal that their folks their residents or folks who are doing data requests in those particular jurisdictions? What kind of publicly available data do they have?

What publicly available information do they have just about the data request process in those jurisdictions? And then as we typically do and as we've done so far for each one of our audit processes, is there literature available or gray literature available that we might be able to make available back to you just around best practices? So those are a broad set of questions. And then we had said that we would engage in three big data collection activities. And so we'll talk about where we are with regard to each one of those data collection activities.

So first one, always we do the big document review. And this one is document and a system review. So the first one, take a look at the city's website, that online portal, and then any publicly available information that is out there. So we've done all of that review. And you all know my lovely colleague Audrey is in the midst of doing all that summarization, and so that summarization process is in motion.

We've taken a look at the GovQA training materials, the process documentation, all the procedural guidance. We spent a lot of time with Nia and with Greg, and they have downloaded and downloaded and downloaded. And we have folders and folders and folders and videos and lots of things. So we've examined all the materials that they have made available to us, and again are in the midst of that summarization. We've taken a look at all of the GovQA performance data that they've made available to us from inception of that system to now when they pulled it, was really the 2025 when all that data was downloaded and are in the midst of that summarization.

And then taking a look at both the model policies and templates from the data practices office at the state level, and then your record retention schedule. So in the middle of that review, have yet to do the summarization of those two pieces. So those two are kind of in progress right now. So kind of, I'd say coming around, if I were to use a baseball analogy, maybe third base as it relates to kind of this data collection and sort system review piece. So getting close on this particular part of the of the endeavor.

Tonight is the World Baseball Series finals. So I think that that is apt for anybody watching, our millions of viewers, just so you know. Starts at seven. Continue. Chef's kiss, well done.

Literature and benchmarking, so our sort of second piece, peer reviewed literature, we've pulled all that. Gray literature, again, Audrey's in the middle of that synthesis. I think it's actually pretty buttoned up. And again, we'll make available back to you like we did before, sort of the table with all of that literature, and then a summary, just for you all to be able to reference in the future. And then we've done public review of all of those other municipalities and jurisdictions, sort of what they have available on their public websites, all the kind of stuff that's available for us to be able to pull in the midst of doing that summary.

And then taking a look at some of those precedent audits that you all had sort of recommended that we take a look at. So that summary we haven't gotten to yet, but that's in motion. And then we had sort of the stakeholder engagement piece. This is the part that we are, I would say, further behind than we had anticipated, just from a timeline vantage point. And I would say that's largely due in part to couple of factors.

One, Wilder has been pretty heads down over the course of the last couple of months, in part because of what's been going on in the Twin Cities. Two, it has been really difficult for us to get responses from other jurisdictions, in part because they have been pretty heads down because of what's been going on in the Twin Cities. So we have surveys developed both for your internal department leads, as well as for your external requesters who've been using your portal. And those are we just did our final review of those surveys today, and so those actually should drop into the field this week. And so we're really excited about both of those being able to go out.

And then we finally had Hennepin County ping us back and say that they were ready to get on the phone or on the video or in person with us for an interview. And then we had a a nice back and forth this week with one of your former city staffers, John McCarthy over at the League of Minnesota Cities, who is gonna pull some strings with us with with some

of

the other jurisdictions to really try to get us in front of folks since we've been emailing and emailing and calling and calling and just not getting anybody to pick up. And I think that is not because those aren't lovely humans in those jurisdictions. It's because folks have just been, I think, overwhelmed for the past couple of months. But the intention is for us to do interviews with each one of those jurisdictions just to talk with folks about their patterns and practices, the systems that they're using internally, and just policies, procedures, the kinds of things that might not be available on their website for public consumption. And then again, John has hooked us up with research leads at the League of Minnesota Cities, who we'll have conversation with to be able to pull information about LMC's guidance related to data practices.

And then we'll also connect with the data practices office over at the Department of Public Administration, or the Department Administration. So stakeholder engagement is kind of that final piece that we'll sort of roll in here. From a timing vantage point, you know, we'll sort of push hard this month in April with our aim to have that draft report in May, so that we can come back in May and do that presentation. As you all know, we make the draft report available. We come and do a presentation.

We anticipate that you're going to give us pretty significant feedback. And so we'll do another pass through to have a final report back to you. I anticipate that final, final report would probably be available in June, since we'll probably do a couple of rounds back and forth. We usually get your feedback, we come and present, we get a second round of feedback, and then we kind of button up that final version of the of the report. So that's what we anticipate as it relates to that timeline.

So I appreciate your your patience, and I really do appreciate John's sort of booster booster power of of sort of hooking us up, and was really excited. Heather and Audrey, yes, of course, I remember all the time we spent together. So it was absolutely lovely. So questions for us as it relates to this process? You were expecting I'd talk about? Updates?

Questions from my fellow committee members, Mr. Lustetter. Yeah.

Doctor. Britt, I was wondering if you know who you contacted at Hennepin County?

I do not know. Audrey would know. And I'm happy to loop back and Yeah, let you know true. Yeah, who in particular it is. Yeah, yeah. But she was really excited. I heard back from Hennepin.

It wasn't you, mister Austin.

No. Shouldn't be.

Jeremy finally responded to it.

No. Wasn't.

It shouldn't

have been yet. But yeah.

Because we had done the, like, the generic and the phone call and the looking for staff and sort of had done, you know, kind of come at it. We'd run up the hill every single way we could in all those jurisdictions. So, but yeah, I'll happily have have us follow-up and let you know who it had been. And we definitely got sort of a, we're sorry. We've had staff out. And also, we've been just

Yes.

Staff. I think folks have been just underwater. So a lot of it is it is not for lack of wanting to respond. It is just because folks have had a lot coming at them.

Yeah. Thank you.

Yeah. This has been a hard time for the important but not urgent things in life. Right. Yeah. I think I saw mister McFig.

Thank you, madam chair. Yeah. Doctor. Britt, I have one question for you. Yeah. How is your team selecting the external requesters for the survey? Or how are you going to be what's the methodology you're going be using behind that? Because I know that's kind of the tricky situation, I imagine. Imagine.

Yeah, yeah. So Audrey and I talked about this. So we have kind of a, you know, all of the requesters that have sort of hit the portal. And so we had thought, oh, let's go for the folks kind of from the last, you know, six months who had hit the portal. We're anticipating, you know, somewhere between a 1015% response rate.

And if I look at the folks who've come in the last six months versus everybody who's hit that portal, you know, there's a few thousand folks who've hit the portal. Some of those folks are from outside of the state of Minnesota. Most of those folks are from inside the state of Minnesota. A lot of those folks are from the city of Saint Paul. And so what we've decided to do is invite everybody who's got a Saint Paul tagged city in their sort of lineup in their address to participate in the first round. So anybody who's hit that portal who's coming from the city of Saint Paul, that's our first invitation pass, and that's gonna be just a little north of 700 folks.

Oh, wow.

So we're gonna see how many folks respond to that and hope that that, you know, response rate looks decently sufficient. I think we'd love to get a little north of 100 respondents to that particular survey. I think we're going to see. If that response rate is low, I mean, again, I think we know how folks are getting invited and invited and invited to participate in everything via, you know, their electronic device. I think we'll see what that looks like.

And if that response rate is incredibly low, and we'd use a modified Dillman, you know, we'll invite, and then we'll give them some room, and then we'll invite them again. And you know, we'll do sort of a, you know, expected pattern of reminders. Then we're gonna open that up to everybody in the state of Minnesota, so anybody who's got a Minnesota address. We're not gonna go outside of the state just because that felt a little bit outside of the scope given city of St. Paul and sort of that focus.

So that's our plan as it relates to those requesters. But when we initially were like, let's, you know, when we thought about sort of the six months and we started to get really, really, really small, And we thought, when we take a look at actually what's a reasonable response rate for somebody that's used a portal and what we expect that percentage to look like, we knew we had to open up our aperture a little bit. And so we thought, alright, let's do a couple phased approach there. Let's not do all of Minnesota, because we're really interested since this is the city of St. Paul Audit Committee in St.

Paul, folks. So we're going to start there and see what that response rate looks like. I'll keep close with Nia so you all have a sense for, oh, it ends up it was 25%. We're great on response, or we really do need to back it out to the entire state. But that's a fantastic question because Audrey and I debated back and forth. And you all had had that, do we go with the, you know, is it really just the high requesters? Is it everybody? And so it was, you know what? We're going to ask, we're going to invite everybody, because you have a variety of humans that are coming through. Some of those folks are representing businesses.

Some of those folks are residents. Some of those folks are asking about personal things. Some of those folks are asking about things in the neighborhood. So we're gonna, you know, kind of open it up to everybody that's got that sort of relevant address. Be that address a business address, or be that address a somebody who's got a residence. Obviously, they have to have an email address. So folks who have the generic, right, they've done an anonymous request. That's not something for whom we have that email. Because it is possible to do a portal request and have it be anonymous, and you can go back and kind of check and pull your data back out. Those are folks for whom we can't actually do that survey. But brilliant question. The researcher in me loves it.

It was something that's been on my mind for the last month. And I was like, I need to talk ask Doctor. Britt this next time I see her.

So I was

kind of curious. It's just me being a little annoying enough to be annoying.

Yeah. No. Great, great, great question.

Yeah.

Thank you. So again, I'm hoping we get enough of a response for St. Paul folks. But, you know, we'll know. And then we will be able to take a look at those two, you know, response sets if we end up having to go, you know, St. Paul and Minnesota. So we would be able to take a look if there were differential kinds of, you know, responses between folks who were St. Paul and folks who were not St. Paul, but were, you know, Minnesota addresses.

Well, thank you.

Yeah. So just because it is an important question. So this week, so anyone from Saint Paul who requested Mhmm. Through the portal in the last six months who has an email address

Yeah. Who's ever requested. Who's ever requested.

Who's ever requested. Who's ever

So we've got a yeah. Few We've got a yep. Yeah. Which ends up being a little north of 700 folks that we've got sort of good Ever. Good addresses for.

Yep. Mhmm. Should get an email requesting them

to Yep.

Take the survey.

Yep. Participate in the survey.

Yep.

Great.

Yeah. Yep. Then I know there were a few entities, sort of other watchdog entities. Now I'm forgetting all the acronyms of them that we suggested. You also talked to COGIE maybe is one of them. Trying to look into my fellow committee members.

The oh, dear.

Sort of good government watchdog entities, sort of.

I know several people are affiliated with one. There's insight. Right? Right. Insight Saint Paul. There is oh, who are the folks that there's an open government group. I'm trying to remember their name.

Yeah. I think we put them in the they were in one of the presentations that you had given us originally scoping it out.

SOS as well, I think. Right? Or it's not different, I'm thinking.

I don't I can't remember if we if we specified them in this one. I'm thinking more like Minnesota data practices blank group or COGI or some group like that. Anyway, maybe you can look back and see. There were there were just some specific entities that we had thought would be good to talk to, and I think that they were in one of your previous presentations. Okay. And if they weren't, let me know, and I can try to dig them up.

Okay. I'll try to brainstorm too. My brain is fuzzy. Admittedly, I've been teaching children all day.

It's the hardest thing possible to do. Okay. All right. Other questions for Doctor. Britt?

Ms. Silworth? Just a quick question, Chair Nager. Doctor. Britt, how can can you tell if there's like multiple requests for at least 700 discrete We

the address list that we've got has been de duplicated. Yep. Yeah. So I don't know from looking at that whether or not it's folks who've been repeat requesters. What I know is it's folks who have made a request through the portal system. Yeah. So and, essentially, the survey is built to say, you have made a request, you know, at least one request through the city's portal system. Yeah. So but I don't have a sense for the frequency with which folks would have made that that request. And we're just asking about their experience with the with the portal system.

And in conversations with Nia and Greg, that's how we built that survey with regard to the portal system. So it's utility, it's language usability, additions or modifications that might make the portal system itself easier for folks to be able to use. So that's the boundaries around which we put it. And we intentionally tried to make it short, so that folks would be a little more likely to do it. So we sort of said, this thing's going to be really quick for you to take.

And we made it intentionally. And as we've talked about before, since we're trying to keep the scope for this sort of in a box, we made all those questions closed ended. So we're not anticipating folks are gonna give us lots of stuff. It's really, you know, if we were to make some modifications to the way that portal runs, you know, could you give us some checkbox responses related to the modifications that we'd make around that portal system?

Yeah. Yeah.

Great. Well, I we have a couple of other items I want make sure we have time for, but are there any final questions for Doctor. Britt before we move on? Seeing none. Oh, Professor Haitlin?

One quick comment. Sorry, Madam Chair.

Or comments, yeah, of course. Doctor.

Britt, just want to say thank you to you and your team for doing all of this on such a short timeline, because I know that it's there's been a lot going on, so I just want to say thank you for that you're doing. And so I appreciate it.

Yeah. Yeah. Ditto.

It's our absolute pleasure. Yeah. I mean, I think I will say I mean, you all know this already, but every jurisdiction in the state and every state agency and every nonprofit is in the same situation you're in, trying to figure out how to handle the volume of requests coming at them about all kinds of data and information. So I think you're I think you're setting the mark, and I suspect there will be lots of places that will wanna take a look at what you're what you're finding after you're through it.

So I appreciate it.

Leading the way. It's where we like to be. Mhmm. Yeah. Thank you so much, doctor Brett. Seriously, we look forward to seeing the next update in May, sounds

like. Yeah.

Okay.

Thank you. And happy St. Patrick's Day. With that we're going turn to our next item on the agenda which is a community engagement update from Mr. McVeigh and I just want to start by saying thank you so much for the hard work you have been doing, the yeoman's work to engage. And you'll see from Mr. McVeigh's memo extensively. I think we've had a community engagement committee for, I don't know, two or three years now. We've always sort of laughingly talked about our community engagement subcommittee of one. But you are a force of nature of one. So I look forward to hearing your update.

A force of nature. Okay. You're really hyping me there, Madam Chair. So, well, happy St. Patty's Day. So let me get started here. So there is a lot that I have to cover, but I will try to be succinct about it, because you all can read, and I don't want to insult your intelligence. But I will first cover a status update regarding all of our district councils. I have actually met with eight of them. It would have been originally 10 at this point, but there was some rescheduling due to one incident with a snow slash ice storm last month with Greater East Side, and also there was an accidental I double booked with a doctor's appointment and with one of the meetings.

I didn't catch until the day before, and so rescheduled with Yanny over at Seiko, actually, to the April. But by and large, there as, madam chair, as you've noted, community engagement has been kind of that's the word I wanna use, languishing perhaps for a period of time. And the last

time we reached out to

our district councils was back in November 2023, and we kind of really we started getting in touch with them all in January. And since I've been hitting the ground running, and there's been a lot of positive feedback. They love hearing from us, actually. Our district councils are so thrilled for us to be meeting with them. They've been really interested in what we have to say and what we're doing.

I've been organizing town halls, and I want to use town halls loosely. That phrase is kind of a nebulous one. It's got a nebulous characteristic to it. And so by definition, a town hall is literally just a meeting where you have officials in some kind of public capacity meeting with residents to discuss various issues. And that actually gives us some flexibility.

I can really tailor these, our town halls, to whether it's a just a line item on a board agenda, and I just come and speak for fifteen minutes. Or if they wanna have a longer presentation of thirty five minutes to forty five minutes, that's also a possibility. Or in the case of last week with Summit University, we had an actual full on community forum, and we went on for an hour and forty minutes. So, which I wasn't expecting that. Half of it was questions, so but by and large, there's been a lot of positive feedback, and the all the executive directors I spoke with are very supportive of our work, and if there's any way that they can support us, they've all said they would like to.

So we are supposed to keep that in mind going forward as a committee. I do have four district councils I'm going to be meeting with at this point so far. Payne Phelan next week, Union Park on the first, Summit Hill on the ninth, and then Saco at the very April. I am still trying to get things arranged with Westside Community Organization, Greater East Side Community Council, Dayton's Bluff Community Council, and Frogtown Neighborhood Association. Having said that, I've been in touch with all four of those district councils and their staff, and there's been really, really positive communication overall.

Actually my good friend John at District seventeen has given us a whole litany of ideas of what we could potentially be looking at as a committee, and I've listed some of them from the district council board members and staff. It runs the gamut from public safety to environmental issues to business regulation and all that, and as well as another big thing being housing, and even cyber security. That is one thing I will admit that I was not expecting to hear a lot from from district council board members. So far, cybersecurity about maybe a quarter of all dimensions are regarding the cybersecurity incident from last summer. And so but a lot of interest with housing, lot of interest with public safety, and so these are just some ideas to get the committees, our collective brains chugging along at ideas that maybe we'd want to refer to Doctor.

Britt and her team on our behalf later on. With in terms of upcoming projects, I had some wonderful ideas from, three different councils that came to my mind, and that were brought to my attention, should say. And one thing that was a concern for a lot of district council board members is that the work that we do as a city, all the different programs and services we provide, it's very broad. I mean, do a lot here in St. Paul.

We do spend over $900,000,000 so that's to be expected. Having said that, a lot of folks don't know exactly what we do as a city. And so one of the recommendations that came from these conversations and these presentations was maybe we as a committee need to have a directory of programs and services by functional area and department that residents can use when they're actually thinking about the different issues that impact their daily lives, and how they can make suggestions to us as a full committee through our survey. And so I decided I wanted to talk with the committee and the staff, and Ms. Vang, Ms.

Romero, if you have any thoughts on this, I do appreciate your feedback, and also thank you both for putting this off, getting this printed off for me. I do I didn't have time to do it at work. So but any feedback that you all have on this, let me know. But I am actually thinking about working on this over the next couple of months, and having it organized. I haven't decided if it's going to be a Word format, or if it's gonna be Excel to make it easy for people to find information. I'm not sure. I have to figure that out yet. I gotta toy with that in my mind a little bit. But that's one thing I'm thinking about work that I will probably work on with after hearing any of your feedback. And then I'm hoping to wrap up our town halls with all of our district councils by the May.

That's contingent on some district councils with their schedules, but hopefully by the May. And when that does happen, I'm going to switch gears a little bit and try to see if we can get more physical materials on the work of our committee out to say with really interfacing entities within the city like our public library and all of our different branches. And perhaps even reach out to our public schools here in Saint Paul to see if there is any way that we can I have a lot of relationships with a lot of schools, so this isn't actually not gonna that hard for me? But see if we can try to have that kind of communication with them about how we can better provide information about our work to the parents and legal guardians of students attending public school here. And I also will be writing to a whole list of civil major civil society organizations that are here in Saint Paul or within the confines of the Twin Cities Metro to see if we can get any feedback that they can provide within our audit survey.

And so I don't expect there to be a massive response from those entities, but I'm going to try nevertheless. And if there's anyone that wants me to do another town hall with them, I'm willing to do that. Would gladly do that and provide it. I educate. I teach during the day and I educate people on what we do here. So it's just second nature for me. Having said all that, I do have some observations for the committee just now that we're at point with our civic engagement this quarter. I've been thinking a lot about this lately. We asked, we're in the world of government auditing as a committee and that basically everything we do is all an exercise in civic engagement. Right?

Because we are providing a process and mechanism to hold entities accountable for how taxpayer dollars are spent, and especially in light of the we have an episode of democratic backsliding going on right now, not just nationwide, but also that's being acutely felt here in Minnesota just because of what's gone on recently here in Minnesota, that ways that we can really uphold those principles that have been the foundation of public life here, I think are important. And that's something that we as a committee actually have a big role to play just within the city of Saint Paul. And so I think where we can be doing more is educating folks on what we're up to. And so for that reason, it really feeds into our mandate and showing up for the community, honestly. And so for that reason, I think that I think it'd be helpful.

I don't know. Jeremy, Stephanie, Dylan, I would love to hear your feedback on this. But I think it'd be helpful if we were to have short autobiographies just on ourselves, just to give a human face to the work that we do as community advisors on our landing page, on the audit committee landing page. Doesn't have to be much. It could be a short clip, but just to get something out there. Along with providing regular physical mailing materials to our district councils, they're really interested in that. I had that come up seven different or no. No. Six different times, and I've been with eight of the district councils. And they thought that would be really helpful because they would actually put that in their materials that they send out to all the readers regularly.

And there was also mention at two district councils, oh gosh, Hamlin Midway Coalition and Summit University, the idea that maybe we should have a blog that just summarizes, like, on a periodic basis where we can have a list serve and send masks, send it out as, like, almost like an email situation where folks can get a update on what we're up to as a committee and projects that we're working on and stuff like that. I am not expecting our staff to do any of this for the record. I

know that

you are all plenty busy as enough. I wanna say I'm here to help you in that regard though. So because I know that

I welcome the help. Yeah. But our website, it's updated throughout the year and Yes. Everything that that the audit committee does is publicly posted. So our agenda would would detail the types of project we're working on.

And I agree with you on that. I I was just I was trying to explain to district council board members, like, we do have a very updated website. Our staff do a wonderful job of updating it. We always have auto reports coming up. It was just they just brought it up to me. It's just like, okay. I can bring that forward, I suppose. Because I mean, it could there's a lot of local auditors throughout the country that do that, so I understand the merits of it. Because often people aren't going to our websites per se. They may be getting information about our work, say, through social media.

Mhmm. And so that may be one vehicle to get a demographic that may not otherwise actually be all that attuned to what we do. And so that's that's the only reason why I actually mentioned it besides the fact that I was getting nagged about it. Not in a bad way. Don't come after me district council board members. I'm not saying this is a bad thing. I support what you're doing. I'm just I'm just a vehicle for conveying information. I'm facilitation. So on that, are there any questions for me, colleagues?

Miss Johnson. A couple of fun questions, I would say, just because now I understand why they were jokingly hitting it a powerful force of one because, first of all, he comes with he comes to the to the audit committee meeting with a memo informed that's probably more substantial than some of the memos that come across my desk. So that's amazing. But I guess just a couple questions of just I know you mentioned some of the themes and things that are coming out of these conversations. But did you notice kind of a variance or like that's depending on where you talked, there was a different level of knowledge of just that we have an audit committee that there's this work is taking place by district council? I'm just curious if you noticed Yes. Differences in that.

Yes. There is a I've noticed quite a bit. There's more of an aware more people are aware that we exist as an entity on the Northwest Side of the city. So from Highland Park, north up to where we meet with Minneapolis and with Falcon Heights. The eastern half or maybe even eastern two thirds of the city are not as aware that we exist.

And when I reached out to a lot of district councils, they were actually kind of surprised. They were thrilled that we exist, but they were surprised nevertheless. And so but they've all been there's been no very positive response. I will say that along the University Avenue corridor, there's been a lot of talk about public safety issues. A lot from anyone that any of the district council board members that direction.

And there's been more mention of cybersecurity from basically kind of I will say actually overwhelmingly in your ward, Chair Naker, and in Council Member Joe's ward as well, that there's been more of interest in cybersecurity, which that might just be because of whatever demographic variables are at play there. So but it's been really but having said that, there's been all sorts of ish ideas that have come up from people all across the cities as as I met with them so far. So Councilmember Johnson, do you have more another question for that? It sounds like you had more.

Well, no, just think that it's I guess I'm not necessarily surprised by that just because I think, you know, folks are going to be learning more about all the different committees that we hold. And I think in community too knowing what resources are affordable to them. I think to the piece around just you know just some of your ideas as well are really great. I think when we have community leaders joining our committees and being able to be a part of it, I know that it can be intimidating to put like your actual face and bios maybe out and about. But I think about how that how knowing a little bit more about the everyday folks that are joining us in this work to give some credibility, I think, to it as well.

Just of, like, this these aren't, you know, just busy committees that there's actually work and there's reports and things that come from it. So I I think that that's great. Folks, there's no pressure, I would say, on putting your bios together, but it's never a bad idea to show that the audit committee is also full of incredibly credentialed folks is what I heard during the introductions. So I think that that's also really cool to see. Yeah. And so and especially on the East Side, I'll just share that happy to, you know, put put out different materials and newsletters and things that we share to boost that awareness too.

Silver told me, and so I'll be following

up with that.

Okay. That sounds good.

I always I've been so fortunate. I do want to add the council members that we have on this committee are always so supportive of our work, and Chair Naker and Vice Chair Coleman have been I've I've appreciate your company coming with me to so many of these district councils. And council member Johnson, you're always welcome to join me at any meeting in your ward if you want. If I understand, you have a we win at home.

We win. Bring

the honorary community advisor.

That's awesome. Thank you.

Other question for Kimmel. Yes. Of course. Are these meeting via MST, or are they required to be in person?

I've been meeting every single one has been in person, except for two of them. One was actually with West seventh Fort Road Federation. They met remotely because that was right when the whole the feds decided to make an appearance. And the other was with Nino because I actually had a doctor's appointment, and so I wasn't able to go I was I wasn't gonna make it on time from the doctor's appointment, so it's easier for me just to plan to have everything prepared at home and work and meet with them from my desk at home. But otherwise everything's been in person. So and I think they're going to continue to be in person.

Thank you. Other Oh, go ahead. I have

an update. So thanks to Noah's community engagement because our survey that is evergreen on our website has stalled. I just checked right now and we have about 15 people who submitted topic ideas.

Oh, that's nice.

Which has

been very quiet since 2023. So it's huge and I will pour through the topic ideas and I will pour it to the committee members to review because that ongoing that work is ongoing. That's awesome.

That's wonderful. And that's such a great obvious outcome that's directly related to your work. I was actually going to say, I mean, only have about nine minutes left, but I would actually suggest that we take some time to continue discussing this really robust memo and the work you've done and the questions you've raised for us in here because in part because, A, we have 15 new topic ideas that I think we should would be great to see and talk about in the context of debriefing the conversation with the mayor. Also, you have identified a whole bunch of potential new topics in your memo. So if we have a few minutes at the end and people want to share quick reactions from the mayor's conversation, great.

But it sounds like we now have a lot of new substance to inform a conversation about topic selection. And I don't want to short trip that either. So other questions or comments for Mr. McVeigh, Ms. Dulworth and Mr. Weston.

I think this is a great. So thank you for for the memo. I think this is a great follow-up to what James was doing. And at the time that he started doing it, it was because I went to a couple of them it was, you know, virtual. And so I think to be able to go in person is is obviously a great next step for this. And so this is wonderful to see how it's how it's progressed from what he had started.

I give a lot of kudos to James because he had a the slide deck and everything. It just needs some minor updates, and it's I was able to hit the ground running with it. I know that he was very busy. He's still very busy as one of the regions at for the university, and so I know that took a lot of time away from doing work here. And so kudos to James and that I'm sad that he's no longer serving with us on this committee.

Ms. Dolworth is right. He planted the seeds and got us off to a great start. And to the extent that any of these district councils, well, many of them had already heard from James, I'm sure when you came back, it was fertile ground for you to be kind of reminding them. So thank you. Mr. Lester?

Yes. Thanks for the update, Advisor McVeigh. It sounds like great work. And I wanted to one of the ideas I think you brought up was to have a list of all the auditable entities, all the programs subject to audit. I think that's a great idea.

It could give people doing the survey a place to start, you know, and something that could even we could link to, you know, for each of the auditable entities, when was it last audited And here's the report. They can see that and, oh, this one hasn't been audited in a while. I will choose this one, you know. So give them, they can choose one of those programs or obviously have something free form that is not in the list they can say to be audited like data practices or something like that. But I think that's really good and giving people kind of a picture of, yeah, what does the city do?

What could even the audits be? Gives them a great place to start. And it's great to hear that there's 15 responses in there right now to the survey. I'm excited to see what those are.

I tickle pink. Forgive me that I'm sure. Go ahead. No. Yeah.

Go ahead.

Oh, was just going to say I'm tickled Oh, okay.

I love that idea of linking to the previous audits too because I think that really helps people contextualize their questions and know that something they were wondering about is something that we've already wondered about ourselves, explored. And they may still want to have further investigation done, but to have that the previous audits be right there, I think it's a really great idea. I don't want to put anyone on the spot, but I will suggest that I think the question about the short autobiography for publication maybe is something that we could follow-up on via email and just ask folks. This seems like a very direct, actionable step, and would love to hear from everybody who is willing to do that. I think it's fine if some folks aren't.

But anybody who is, it would be great. I agree to have those up there. And I just wanted to ask about the creating mailing materials. My first thought goes to, oh my god, it's so expensive and hard. I agree. My second thought was, could it be emailing materials, in which case, could it be something like writing up a brief summary of this meeting and linking to it that goes out to district councils after these meetings for them to put in their newsletters or elsewhere just to say this is what the ad committee is working on. Trying to think about minimum viable product given that we're limited in staff capacity and but I hear the point about wanting to be communicative.

That's what I was leaning towards just because our staff are got enough going on as it is. We don't need to put any more on you, more necessary. And so I think that if we can have someone working on this, I think that would be I think that'd be easier than having to meet reading materials or physical paper materials because otherwise we're going to be blowing up our budget.

Okay. Discussion, questions, comments? And then I, Mr. McPhray, maybe you can summarize the next steps because I think you had you mentioned having some maybe that list ready for a review in August, you were thinking, or?

Yes, I will get that direct I should have that directory done by August, if that's something that you it sounds like we're very keen on. So I will have that done in by the August. So we should have it for certain at our September meeting. And then let's see what else. I have more that I'll be bringing next month because I've just been working on a literature database. So if not next month, if we have too much going on, we can always save it for me or later on.

Well, thank you, I think, on behalf of all of us for this work, seriously. And to the extent that we can be helpful, Council Member Johnson and mister Coleman as well in helping connect with any of the remaining district councils. I see some them are in my word. Please let us know.

We've been in touch with all of them. We've had conversations. So it's just fine getting it's just figuring out a schedule, basically. Basically. They're trying to figure out their schedules right now. Okay. And so don't worry.

Okay. We'll be we'll be taxed.

They're what they're they're behaving themselves.

Good. Well, thank you. And I know they have a lot going on as well. They do. No knock on that. All right. Well, then with our very with our last I thought this would happen. Two and a half minutes. I'm sorry. No. No. No. Not at all. I wanted to take time for that. And this is really remarkable. So thank you. But while it's fresh sort of in our minds a month ago rather than two months ago, any just quick and again, we'll have a more robust conversation about this next time. But thoughts on our conversation with the mayor and all four assistant mayors, which by the way has that's the only time that that has yet happened since our new administration has come in. So that really was a huge honor, I think, to this committee that we got that level of attention. Mr. Lester?

Yeah. Chair Naker, that was I just wanna say that that was wonderful that they all came. It was great to meet the mayor. And the one thing I wrote down, so a takeaway of, I think it came up College Bound St. Paul as a potential audit idea. So just wanted to put that out.

Thanks, Mr. Ollie.

I mean, Mr. Ollie Stitter.

Mr. Donnell?

I totally agree. I really give the mayor and the team a lot of credit for coming. That was really important, and I really appreciate that. And they seem to sort of understand the spirit of audit and it didn't seem punitive. It seemed like improvements. So I thought that was a great basis. I think we have work to do to put the topic for what's next because that didn't come up. We talked about Collegebound and I flavored that point too, but I think there are lot of other options in addition to that we could consider. So glad they were here and I think there's work to do to sort of narrow what we can be doing moving forward.

Thank you. Yeah, I echo what he's saying. I thought it was took it was amazing that she wasn't scared of audits. And so and she actually embraced audits. And so that's really to be a good spot to be when trying to work with the administration.

Right. Yeah. And the interesting thing about College Bound is I think about it and I agree, Mr. John Lee, there are lots of topics we could explore. It occurred to me it's an interesting sweet spot in that with a new mayor, you know, you might sometimes be there might be a downside of studying things that were done under the previous mayor because there's there could be a tendency to sort of write it off and not really own it as, you know, that was the last administration and we're moving forward or, you know, or even to dismiss the work too quickly if we had negative findings or really difficult findings because it wasn't, yours.

In this case, this happens to be a program that the mayor helped create and is a program that's been around long enough now for her to for us to be able to audit. So it's actually kind of in a sweet spot of ownership even though this is a brand new mayor and having had time to demonstrate results. So that is interesting. Mister Rafay, were you gonna add something? I thought I saw your

I lost my train of thought. It's all good. No worries, madam chair. Okay. Maybe they'll come back to me. Sometimes they come, sometimes they go.

None of us are disappearing after this meeting, so you know where

to reach.

Well, I think my suggestion would be, if there's no other debrief thoughts, that we get the topics that have been submitted, that folks take a minute to look over Mr. McVeigh's summary. And if there's any color you want to add by email, specifically by way of sort of to Ms. Johnson's questions about how often a topic came up. You mentioned cybersecurity came up a lot, but otherwise it's not clear from the list like how weighted those were.

So if you want to if there's anything that you want to provide later to say, you know, these things were particularly frequent. But I think if we can review your list, review the 15, think about our conversation with the mayor, and then come back to our next meeting in April prepared to really look at come up with a lit oh, we we not meet in April?

May. We might want to

talk about that just because it sounds like in May we're now going to be getting the update from Doctor. Britt. And with only an hour, like, we don't have it, so we'll talk

But about

prepared to look at a list and talk about our process from there. It sounds like we're not gonna need to select a new topic until July because this current audit will go through June, so we have some time. But as all of us know who've been on this committee for a while, we don't select a topic overnight. Overnight. Sometimes takes half a

year. So

how's that sound to folks? Yeah. Great. And we'll get those lists out well before the next meeting.

Yes, ma'am. Do we have like a list of Amir's suggestions?

Believe we only really talked about college ballots.

That was the specific one.

That was

the one that she named. But I believe she was interested in understanding programs that the city implemented and whether or not we took into consideration the budget impact of implementing a program that a mayor or the council adopted. It might be a good

idea actually in the saying if we followed up with the mayor staff to say that was a great conversation. Were there any thoughts about specific audit topics that occurred to her afterwards? Okay. Just so that we can get any any more that she might have to say. Okay.

Yeah. I think as a follow-up to that, I was just I definitely will take a look at some of the listed the listed different topics that are here. And I know that there's a couple of East Side folks, Osiko and Dave Bluff being in in my ward too, so I'll flag that for them so that way they can provide just a couple of topics of interest for them. So I see a couple from board seven on here so I

can help with that. But then also just take a look and review and but yeah.

If there was additional add ons, just you know, folks have the one, but I just was wondering if there was something else that we should consider too. Mhmm. And or, I guess, maybe taking this list that you've compiled

to and showing it to staff and getting their feedback. And I will plan to bring next time as well, when we talk about the topics, also our process that we've used in the past to kind of narrow from a big list to a few, to do a risk analysis, to run them by staff and the council. There's a number of steps which we have in pencil and sometimes rethink. So I'll bring that process as well next time.

It's iterative, but it's fun.

It's iterative, but it's fun. That should be our timeline. It's not a committee.

There we go.

Great. Well, with that, we are two minutes past time. I want to thank everyone for a productive meeting. As always, we will see you probably in April, but we will connect about We'll schedule that.

Yeah. So we typically meet the third Tuesday for our audit committee meeting. So I'm looking at the twenty first sorry, at the twenty first if folks are available at 04:00. And if not, then I'll try to find a different time.

Great. Please, does anyone right now know you're not available on the twenty first to 04:00?

The April 21?

Yeah. I'm not. You're not available. Anyone else not available on the twenty first to four I

don't know.

You don't. Okay.

We'll send it out via email. I thought it might be easy to do it here. Right. Thanks, everyone. We are adjourned.