Budget Committee - Regular Meeting

The St. Paul City Council Audit Committee to order. Roll call please.

Committee member Dilworth, virtual committee

But let's let her stay here.

Oh, sorry. I

think we can't hear you, Stephanie. I think you're still mute.

She's not muted. Oh, you're not on high enough from the screen.

Hold on. Let me go check.

Maybe call everybody else first.

Okay. Committee member Gloucester?

I'm here.

Committee member McVeigh?

Present.

Chennaiken? Here.

And, council member Coleman? Here.

Perfect. So we're just waiting on

I still can't hear you, we're seeing what's wrong. You can hear us, I take it? Looks like yes. Okay. Hopefully it's on our end.

We'll give it a second and then we'll.

Need to face the mic towards you. Oh. So it picks up your voice.

Oh, sorry.

Okay. They're working on establishing audio for for Stephanie.

Stephanie Okay. Okay. So we don't have it fixed yet, but

We'll just acknowledge that

Stephanie But Ms. Dolworth is here and can hear us. Okay, great. All right. Well, welcome everyone to our November Audit Committee meeting. A lot has been going on and we have a lot on our agenda, so we're going to dive right in. First we're going to welcome up Doctor. Heather Britt to give us an update on the data practices study scope of work, which is our not current audit, but our about to be current audit. Ms. Vang, did you want to add anything before?

Ms. No, I just want to say that this is our third study of Wilder Research, head by Doctor. Britt. Welcome up.

All right. Let's see. Thank you. Thanks for having me back again. Nia and I and your legal team and other folks the city have spent a lot of time together since the last time I was with you. And we have done some tuning and some modification of the scope of work. You all have given some additional input, and so this is really just an opportunity for me to sort of say, here's the revised scope for us to get kind of a final stamp of approval so that we can really get earnestly after it. So that's our plan for today.

Alright. Here we go.

Let's see if we can

Did he go to sleep? No. Didn't chew in there.

There we go. I got it. So we're just gonna do basically the scope update, sort of that refresh. So again, just kind of a reminder of our purpose and we have tuned our purpose. And again, this is really in response to some of the feedback that we got from folks across the city, and that includes folks from the mayor's office.

It includes a variety of members of the legal team. So to assess the effectiveness, efficiency, and compliance of the city of Saint Paul's data practices, request processes, tools, policies, and user experience with a focus on the city's use of GovQA, which is really the sort of management system for those data requests and its alignment with the Minnesota Data Practices Act requirements. So that's kind of the sort of the refreshed version of this purpose, which feels, I think, comfortable enough for everybody at the table so as to give some input information to the audit committee, so as to give input and information to the city. But so as not to present any additional risk. So I'm going to do what I have done sort of previously.

I'm going to talk about some key research sort of questions or key research areas. I'll talk about sort of the key stakeholders. I'll talk about kind of that data collection approach. And then I'm to talk a little bit about our adjusted timeline, given it's November, and I think I came two fifty two years ago the last time around to talk I'm about kidding. So I came in the past, and we've done some modifications. In the first sort of area, so you'll see A1, I'm really just talking about kind of area one. So we really said this talks about GovQA. Again, that's the tool that the city is using to do all of this kind of tracking and movement around of what those requests look like. The processing, tracking, and the movement and the closure. The communication with end users.

So talking about its capabilities and its internal use. So key questions here. What does that standard capability set look like? How has the City of St. Paul configured or customized? What GovQA looks like for its purposes? How is that use documented for city staff? So what do those procedures, training materials, process maps look like? And does that GovQA system provide sufficient management controls to really efficiently and effectively process data practices requests? So a little bit of a kind of, you know, is this tool sufficient for the needs that the city has?

So that's sort of the kind of tool capabilities portion of how we're thinking about this work. The second part of this, reporting and data trends. So if you remember, this was one of the things that you all were really curious about. So one of the open questions, what kind of reports are available to the city staff through GovQA? Which reports are vendor provided?

So some of those reports were sort of out of the box reports that just showed sort of up for the city staff. Which were created specifically for the city. And then what do aggregate GovQA reports reveal about request volumes, trends, timeliness, and patterns since GovQA's implementation? So, and credit to Nia, who has already started, to dig in, have conversations with the vendor about actually getting all of that stuff sort of pulled and spun up for us. And again, I will say we are looking at aggregate.

We are really not interested in, for all kinds of reasons, looking at any line level information here for the purposes of this. So we moved into kind of this, and I think one of the pieces of feedback that you all have received since the last time we were here, and one of the pieces of feedback we have received directly from some folks in the city, is about just some of that resident facing experience, and so wanting to focus really that resident facing experience on the portal as a way to kind of narrow what that scope could look like. So thinking about the portal that residents use actually to sort of request data from the city. So what online portal do city residents use to submit data practices requests?

How does

that portal function? How do residents receive correspondence and data from the city? What is the experience of residents or end users when navigating the online portal? And then how do frequent kind of low need, high use requesters? So we have a particular set of users that we have been pointed to perceive systems have compared to other jurisdictions that they routinely interact with. So really just getting a sense for how is this user experience in comparison to their user experience with other jurisdictions.

Sorry, miss Coleman. Thank you. Just a quick question on that. Yeah. Do we anticipate that our low need, use requesters are making frequent requests in other jurisdictions?

There may be some. Okay. Yeah.

Thank you.

And I don't know if we will go as far as to name the individual, but we do know of a few names that do that. Okay.

Great. Yeah. Thank you. Just a curiosity question. Yeah. Great question.

And then policies, training, compliance. And this is one where I would say we are trying to be particularly pointed. So what training programs and guidance are provided to city staff responsible for processing data practices requests? Are these training programs and guidance adequate to ensure compliance with the Minnesota Government Data Practices Act? How does city training programs and guidance compare with model forms, procedures, and guidance issued by the Minnesota Department of Administration's Data Practices Office.

So again, we're focusing really on that kind of training piece, that guidance piece, that part here in this sort of particular kind of section, Section Four. And then this sort of final area is really that sort of comparison with other jurisdictions, which is something you all are really interested in. And this is a place where the mayor's office and your legal teams had a whole lot of comfort and sort of a broader sort of sense for latitude here. So doing some work finding out what do those processes, policies, procedures look like in other Minnesota jurisdictions. And so we're gonna take a look at Hennepin, Ramsey Counties, Minneapolis, Bloomington, Duluth.

What kind of request management tools and online portals do they use? So just to get a sense for what does it feel like to do that request online. And then what kind of tools are they using internally for doing kind of that traffic control management. What publicly available data exists regarding their request handling? So we're not gonna go so far as to say hand me over your internal data. But we are gonna look and see what sort of data are publicly available. So logs, statistics, performance measures, anything like that that might be available. That might be a bit of a model for Paul to be able to take a look at. And then as we alluded to before, gonna take a look at kind of that peer reviewed literature and then what we call gray literature. So just kind of best practices literature.

What does it look like for other jurisdictions across the country? Exists for kind of the municipal public records programs nationwide. So taking a look kind of what else what else is out there and what might be good. Additional guidance frameworks support for the city to consider as you're moving forward. So those are kind of our big areas of focus.

Again, those have tuned a fair bit since the last time I was here for all the reasons that I suspect you understand. So All right. We do have a much wider and deeper set of specific names attached to this, but I'm gonna talk at kind of a high level about the stakeholders. So obviously the city clerk's office, you all did a fabulous job celebrating her. So I just wanna say that for the record.

It was fun to see that in the paper. Yay, yay, yay. Sort of council committee staff, obviously. We'll be sort of doing that door knocking. And then frequent requesters to the Department of Safety and Inspections through the city online portal.

So we'll be sort of requesting, and we have in hand the names of some of those humans. And then sort of state and external entities, obviously the data practices office, the state auditor, so taking a look at sort of what are those regulations, what are the what does that guidance look like. League of Minnesota Cities, subject matter resource expert. And then as we alluded to, those other Minnesota jurisdictions. So wanting to kind of understand what do those communities or counties engage in from a sort of sort of best practice or sort of what are their are their plans, policies, procedures, guidance, online portals, and sort of data tracking systems that they use internally.

And Doctor. Burt, before you keep going, I know this was something that we I got a lot of feedback on in terms of who should be consulted as a stakeholder. And I see in this handout that we have that you have sort of written under key stakeholders that the Audit Committee has agreed members of the public should be included as key stakeholders and that consultation with the Data Practice Office and Minnesotans for Open Government will strengthen the scope and methodology. I see the Data Practice Office written here as a specific entity. Don't see Minnesotans for Open Government and I'm not sure I understand is frequent requesters to Department of Safety inspections through the city online portal the same as members of the public who have made data practices? Period.

They are. Yeah. Yeah. So I think we're thinking about that as kind of the subset of members of the public that we're going to sort of focus on, kind of those frequent requesters as folks that we're going to want to get a sense from. Tell me about that experience of using the online portal. Tell me about that experience of using the online portal. If you've used other online portals, how does the City of St. Paul online portal sort of compare? Tell me about the sort of responsiveness, timeliness, those kinds of things. So that I think is kind of

that

public feedback. And we did decide we're going go with a survey there as opposed to sort of a more open ended qualitative set of feedback for intentional purposes. And could you state your first sort of question first?

Also, just so Department of Safety and Inspections is that's regardless of what you're asking for data about, that's where you go, Ms. Vang?

Yes. I

just want

make sure we're not limiting it to this.

When we pulled the names of people who commonly asked or what I call frequent requester,

found

out that a lot of the request was routed through the Department of of Inspection. So they were the frequent folks that we identified. So perhaps we could also look at other department, but these are the ones that have the highest volume that came in.

Yeah, mean I think I guess as long as we're doing a survey I'm not sure why we necessarily only need to talk survey those who frequently requested. It might be interesting to cast a broader

We could.

Broader canvas and I don't want to limit it. I don't see why we would limit it to asking of one department. Female You know, it seems like we're trying to understand our overall data project. Yeah. Wasn't intentional Female

to Chair Naker. It's just that when we pulled up the names, it just happened to focus on that department. So we will try to expand that if that's the case.

Great. And then I just was pointing out that I don't see Minnesotans for Open Government in the list. I'm just noting like there's a paragraph and then there's a list and I want to

make sure I understand. Yep. Yeah. Yeah. So we can do the alignment of the state and external entities with that paragraph. Yeah. Yep.

Yeah. Okay.

Thank you. We can keep going.

Yeah. And I think I left the paragraph in from the modified scope that came from you all. So Yeah. Yeah. I can do some some syncing and some tuning. So I assume that was language straight from you all. So it's like,

oh, I'm gonna leave that in. So Okay.

Yeah. And, yes, we can you know, we're gonna do an electronic survey platform so we can talk about how why we wanna do that and the period of time that we wanna think about how long ago those requesters Okay. Sort of used that online portal. So I don't know that we wanna go back as long as GovQA has been in existence, mostly because we actually want folks to have some recency about that experience. So let's connect and think about how, you know, how wide we want that from a survey window vantage point.

So, but we certainly could go with a wider capture of humans. Let's see. Yeah, so we've got sort of a variety of ways that we're thinking about sort of the review. So that kind of document and system review. And so you can see kind of the review of everything that sort of the city's got.

NEHA's been starting to assemble all the kind of training materials and all the kind of standard out of the box stuff that is coming from GovQA. Taking a look at all that sort of performance data. And then kind of taking a look at kind of what are the model policies, sort of the record retention information, kind of the standard things that we want to be taking a look at. Just so you all have in hand, kind of here is the standard set of guidance that would be the standard set that you might be thinking about and sort of considering. Especially as the city is sort of assembling that new team to really sort of think about addressing your sort of data practices and how you're making sort of folks across the city feel like you are as responsive as possible in a way that feels as supportive as possible given, you know, what is really an increasing increasingly wide and deep volume of requests coming through to the city.

Know, taking a look at that literature, taking a look at some of the sort of benchmarking. So again, like we said before, kind of we'll do, we'll have our library services team kind of take a look at this. You know, and take a look at, and we've already started doing some of this sort of review of what those publicly available sites look like for municipalities across Minnesota. And then taking a look at, you know and again, appreciate you all sort of referencing and sort of sending this our way. Kind of there is already some public precedent for the kinds of standards that that the city might be thinking about.

And so making sure we're just tapping into what those standards look like. So the stakeholder engagement, and again, we'll think about kind of doing, broadening that survey just from a volume and an end vantage point. We've already had a couple of conversations with Nia and, you know, former city clerk before she departed. So thinking about some additional conversations just to just to make sure we have a really good handle on exactly what that process looks like. And then thinking about additional interviews with some of the staff who are responsible for kind of that data request process.

So Nia's been great at identifying who from each one of those departments is responsible for that work. And again, we are going to keep conversations inside the scope of what this of what this particular audit is focused on. We will do interviews with folks from each of those Minnesota jurisdictions just to understand what their processes look like. And then again, have that consultation with League of Minnesota Cities and the Department of Administration, that particular office. So again, just to understand best practices, guidance.

So really just to get a handle on kind of that, in order to really sort of develop for you all, kind of here's the book of best practice and guidance that could be the reference point as that new team is sort of put into of into deployment. Timeline. We have just kinda shifted it forward a little bit. So kinda we're in that scope finalization. We're gonna step into that. And we've already started some of that, like document and data review. Starting to do interviews. Develop that survey. We're going kick off that literature search. We'll keep digging into analysis, really kind of wrap up.

We hope those interviews make sure that survey is kind of wrapped up in the field. We're doing that synthesis of literature. Have that draft available in February, and then, you know, be able to present that final report in March. And again, we'll do the same thing that we did before. You'll have a chance to take a look at the report, give feedback, we'll present, you'll offer additional feedback, and then we'll do kind of a finalization of of that report for for you all. And then I think again, once that team is on board, comes on board, has already been on board, happy to do additional support with that team depending upon what is needed given their feedback and needs that they may have.

Great. Thank you so much Doctor. Britt. I want open it up for some brief questions. We do have a ton on the agenda, so I want to see if colleagues have questions, comments, concerns. But this is going to be whatever this is our last opportunity for feedback before the audit actually begins. So Mr. McVeigh.

Thank you, Madam Chair. Doctor. Britt, with regards to data collection approach number three with stakeholder engagement, I just wanted to make sure that our consultation does include the state auditor's office as well because when they are auditing our financial statements every year for the ACFR and the CIFA for federal programs that sometimes they are actually going be looking at our compliance with the Data Practices Act, sometimes with certain federally funded programs in particular. And it might be helpful just to reach out with OSHA while you're doing your field work potentially. I just want to make sure that they're included in that consultation. I know they're one of the state entities that's listed, and I just wanted to bring that up. Okay.

Yeah. Yeah, yeah, yeah. I think we again, I'm going to put edges around that. So, happy to seek their guidance edges Thanks, Rachel. Consultation that way. What we won't step into is any of the, like, feedback or any of those kinds of things, right? So we're really looking for kind of the expert here's the kind of general consultation and guidance that we offer to all cities. So, yeah. But yes, happy to add

them. Okay.

Yeah.

Thank you.

Yeah. Yeah. Yeah. Great suggestion. Thank you.

Mr. Allsiter.

Yes. Chair Nacre. Just a question on the survey with our external requesters. I do agree with the approach that it should be broad and hopefully we get infrequent and frequent requesters. I did wonder with the frequent requesters, do we think we understand why they're requesting frequently? Or is that something that this survey is hoping to bring to light?

It's certainly something that we can include. I suspect that some of the staff have a sense for some of that. That's what

I wonder if we already know. I personally don't know. I would maybe like to know.

Yeah. I mean, I do think it might be helpful for and we talked about we're going to keep that survey entirely closed ended. Yeah. And so I do think it would be something where it would be helpful for them to give us in their in those sort of closed ended responses, their reasons for, you know, for those requests.

So And maybe how many requests have you made in the last year or something, you know?

Yeah. So what kinds of requests, number of requests, right, you know, just kind of that. I I do think kind of, you know, perception of that portal, usability of that portal, right? You know, kind of the the the kinds of basic sort of user experience questions that I think we want to get at. But But I do think from their lens, because we should be able to pull some type information from GovQA. Again, I'm conditioning that, because the back end of that system, as Nia has told me and I am discovering, is a little trickier than I think

what Okay.

I was hoping. So but she has been living this.

So When you look at the aggregate, are you gonna be able to see how many requesters and how many requesters or for each requester, how many requests they made? You can de identify it, but just say how many frequent requesters do we have? How many one time requesters?

It's kind of a weird answer. If they create an account, then you can you have to search if you have a known person. You can search to see the type of request they made and how often they requested. If you don't have a named because you can be anonymous.

You can be anonymous.

Yeah. And we get a lot of those too, but you can't tell.

Yeah. So Okay.

Mhmm. Learned something new.

Already tell we're all going learn a lot about

this process.

Yeah, Just from saying the scope of the audit, not even auditing. Did you have more Mr. Austin?

No, that's yeah, that's very informative. Thank you. Yeah.

All right. Any other discussion? Otherwise

Just one.

Female Thank

you, Madam Chair.

I just

want to say thank you, Doctor. Britt. I'm really excited about this audit and this revised scope is just wonderful. So, I'm nerding out over this order.

Good. Good. I wanna say thank you for your patience. Thank you for inviting half the city to get together to help advise us on how to negotiate, getting to a place that feels, I think, comfortable for everybody around the table. So that was not an easy set of conversations. And so I really do appreciate your patience and your support and your support in gathering all of those humans together. So I think you're at a good spot. I think it will really especially be supportive as that new team has sort of stood up. And supportive for the staff and thinking about, what kinds of opportunities are ahead of you.

Thank So. You so much. We look forward to the steps ahead. And by the way, Ms. Dilworth, online, if you raise your virtual or real hand, I'll know that you I just want make sure you know, you're always able to indicate that you want to talk and then keep an eye on you. Awesome. Thanks, Doctor. Britt. So as we move on to the next topic on our agenda, kind of we've just launched our current audit. And as you know, if you look at our audit committee, handy dandy complicated yet helpful schedule.

Now that that is underway, we kind of move into opening up and looking for our next topics. And as we've talked about last time, what we're hoping to do this year is have sort of one big idea gathering phase, which is now through December. And then choose our two topics for next year so that we're not running a twice a year topic engagement or topic farming, harvesting process. We're doing that once, choosing two, and then going through the year based on those two. I so that's why we're moving to community engagement now because this is the phase in which we are soliciting those topic ideas for the harvest.

And I want to say a huge thank you to Mr. McVeigh who has ably stepped in where Mr. Farnsworth has stepped out to help us with the community engagement process. We talked about some ideas last time which I think Tony and Nia very beautifully summarized here when we were discussing how should we get the word out about what we're doing and about the topics that we're trying to gather. But I want to turn things over now to Mr. McFay to share the work that he's been doing and lead this part of the discussion.

Thank you, Madam Chair. I will begin by saying I reached out to the district councils that did not respond to our prior correspondence. I mean, I should back up a bit. The council or the committee itself reached out to all 17 district councils back in November 2023 is my understanding, long before my time. And many of them did respond and participate in the topic selection survey for residents.

There hasn't been that kind of engagement with the district council since then. Six of those district councils never actually returned any correspondence with our committee. I took the liberty of reaching out to those three district councils or those six district councils. Three of them have responded to me at this point, either over the phone or through email correspondence. They are the West seventh Fort Road Federation, which is District 9, Como Park District, Como Community Council, District 10, and District 15, the Highland District Council.

And I appreciate those folks that were kind enough to return my correspondence. I have meetings with those three district councils next month, actually, with Como on December 17, and then again with Fort Road, West 7th on January 12, and then again January 8 with Highland District Council. Unfortunately, I've not gotten any other communications from the other outstanding three district councils. When I found out that we hadn't been in contact with any of the other district councils since November 2023, I decided it might be a good idea to also get in touch, follow back up with the other 11 district councils and the response rate has been, to put it gently, lackluster. I did receive a phone call from the executive director for the Union Park District Council, and I'm we've been playing a little bit back and forth.

I'm hoping to get in touch with her sometime this week yet to find time to meet with her board potentially at next month, if not in January. Otherwise, the district councils have not been very responsive, and it might be partly because I'm sort of a nobody. I as a community adviser, it would be

very of this audit committee.

I am an I am a of this audit committee. Yes. But I don't have, say, the weight that a council member does. And I was wondering if it might be a good idea if to speak one on one with the council members and see if any of them be willing to reach out to the in their staff to the some of the district councils are being a little non responsive and see if we can kind of get some communication going. Obviously, I know all the council members are very busy and I'm here to try I delight in workload a little bit for you.

So I'm more than happy to still be going to all these district councils and meeting with them and informing them what our work is and trying to get the public to attend these town halls during regularly scheduled board meetings and see if we can get more participation, excuse me, with which is getting more audit topics out there because obviously if we have a larger data set to work with, the better we can actually be at having more timely decisions on what kind of audits we want to have and that we can recommend to the full city council. And so I did reach out to Insight St. Paul and to St. Paul Strong, and both those organizations did actually complete the survey. The steering committees for both those organizations were very interested in her work as an entity.

And so actually ecstatic was the response I would categorize it as. It's nice to be loved. It is nice to be loved. But other than that, that is the extent of my outreach at this point in time. I do hope to continue this. This is gonna be an ongoing thing. I imagine that I will be having meetings, like, not this month at this point, but December, January, and perhaps even into February. And I know that doesn't exactly line up with our desired timeline. So maybe I need to fudge it a little bit, but I will keep working on it.

Thank you again for your leadership on this. It is not easy work, but it's such important work. So thank you. I saw miss Coleman. Thank

you. Thank you. I will echo the thanks because it is it is not easy, but it is so appreciated. And I apologize because I got distracted comparing these dates to my calendar. So I'm having some listening struggles. Is it helpful so just thinking about the district councils, I have five district councils in my ward. So super happy to make sure that this is on the radar of all of them. Two of them are on this list.

Yes.

Is it helpful for like, these meetings are that are on listed on here, are they're just normal community council, district council meetings? Yes. It's like, should I show up at the Como Park District, like, the d ten meeting tomorrow and talk to them about audit committee? Is that helpful?

If you want to. Here's the thing. They already have the the meetings that you see that are at this point in time that are on the calendar. Those meetings are full. They have full agendas for those evenings. And so that is why I will not actually be presenting until next month at the earliest with the with Como, for instance, Como Community Council. But with the other district councils, it'll be in January at the earliest. And that's why. So you have if you you are more than welcome to ma'am, if you wish to do so.

I I don't know how appreciated it will be if their schedule's already fully packed, but just wanna offer that if it is helpful, especially with the with d 10 and d 11, I'm happy to take those meetings off your plate, be the one to go or, you know, just coordinate or what whatnot. And then same goes with the other three district councils in Ward 4. Happy to connect with them, make sure that this is on their radar. And if it's also helpful to, like, loop me in on any conversation, happy to do that. And then, you know, since we only have three council members on this committee, I know that leaves a lot of district councils uncovered.

So maybe the council members amongst us can talk about how best to make sure that our colleagues are committing this information to the district councils that they represent because I think it's you know, I'm happy to go over to the East Side, but I feel like they would prefer to hear from council member Johnson and Yang, for example.

Mhmm. Council member Coleman, if you would like to come to any meetings I will be attending there in your ward, you are absolutely more than welcome to join me. I would appreciate the company. Awesome.

And Mr. McVane, maybe I appreciate that, Coleman. Maybe what would be helpful would be for you to sort of serve as air traffic controller. So, you let us know who you haven't heard back from. And then to Ms. Coleman's point, the council members can figure out, okay, how do we need to reach out to them in a different way perhaps to get their attention. And then depending on if you're not, you know, I that 17 is a lot of district councils to go to. I think you should also feel free to deputize us to to attend some of the meetings as well. So

I will most likely be doing that.

Yeah. Yep. I think you should you should do tell us what to do, but you're owning the organization part of it is huge.

Yes. Great.

And then thank you for reaching out to Insight and Strong. Yes. I'm looking at the other ideas that we had come up with last time. I am wondering if so, Nia, you us? So the topic selection survey, is that that's always live? Yep. And did we receive like a Facebook blurb or something for us all to help spread the word about it?

I believe I sent it to all the committee members and advisors on this. So if you didn't get a copy of it, I'm happy to reford it to them. I give you guys three options. You don't have to use all three. But if you want to, by all means please help spread the word.

Okay. Yeah. Can you maybe resend that to all of us? And I think all of us should make an effort this month to put that out on our social medias and our platform to platforms to let people know that we're soliciting topics. What other ideas did folks have besides the district councils, besides some of these specific community groups and besides all of us advertising the topic selection survey for that gathering in.

I know we talked about talking to the city attorney's office. I think we're in a different environment now that we've had a mayoral transition. So I think staff is a little bit preoccupied and uncertain, rightly so. So it may also make sense for us to hold on some of the conversations with staff. Also may make sense for us to talk to our incoming mayor and her administration when they're set up.

We may we might need despite our our virtuous attempts to make this the calendar that always stays the calendar that we never veer off from with a new mayor coming in, we may wanna take some time to find out from her what she would like to do and what she is interested in studying because I think there may be things that she's very interested in learning more about at the city. But what other ideas did folks have in terms of ways to get to solicit topics if they're not already captured? And and specifically, there's any ideas you have that you yourself can help with because I think it's all hands on deck. I can. Yes, mister McFay.

Thank you, ma'am. So in Seattle, their city auditor's office, they actually have an ongoing what they call an audit fair. And

if

I recall, they go around to their they actually participate in, how do I put this? They go around to schools within Seattle and actually talk about the work that they're doing. And the reason for that, from my understanding, is a lot of public schools in Seattle are used as polling places. And so they decided that it'd be since a lot of folks in Seattle are already kind of affiliated with a given voting precinct and their kind of schools are central in their community in that sense that they go around to school districts or not school districts, but individual schools throughout the year and allow the public to come in and talk with them. And they have these audit fair basically is what they call it.

But it's in to be all in all fairness, it's really more like a town hall. That's something that could be done. I don't know if we actually have the bandwidth to do that here with our school district perhaps. The other something else that I had that came to mind is oh, a little bit of a, that train just derailed a little bit. Maybe hopefully, that comes back to me. I forgot what I was gonna say.

Alright. I'm sure it'll come back to you. We'll we're not going anywhere.

Yeah. Exactly.

Thank you for that suggestion. And I couldn't see if I saw a hand or not. Okay. I love that idea. It does seem like a high capacity sort of project, but but it's good to add to our list for such time. And I'm almost wondering if there are I don't know if there are newsletters or other ways of communicating with the students, whether it's St. Paul Public Schools. I think it probably does require more of a presentation, though, because otherwise you something in a newsletter, have no idea what the what the audit committee is.

And they do give a presentation too and it's very very broad. It's for general public so they try to gear it in a way that kids will enjoy it too.

Right. Right.

So We

need to see what that looks like if you can ever find the link to it, mister McVeigh.

I'll dig into that. I also actually have to I before our committee meeting, I was talking with miss Vang a little bit, and I need to still gather I was gonna share some literature regarding civic engagement and public audit with the committee. And I promised to do that months ago, and I still haven't gotten around to it. That's that's on me, my little screw up there. So I do apologize to colleagues here, but I will get to that at some point. So I will make sure to include that as part of that paperwork dump.

That sounds good. You've been busy, so not to worry. Okay. So it sounds like we're talking to district councils. We're gonna send out our topic selection survey. We have already reached out to some specific organizations. We I also know talking to council members and their staff. And I think maybe council member Coleman and I can discuss how best to talk to our colleagues to get their ideas, especially post budget process. That's often when people are very much thinking about, like, these are things that I really wanna dig into next year. All with the goal of coming back to our December 16 meeting with a whole bunch of topics, which we would discuss and try to narrow to four for risk analysis.

That's our goal. And I think, mister McVeigh, if you're continuing to engage district councils beyond that, that is perfectly fine.

Okay.

This is ongoing work, and the more that they know about us, the better. So, anything else to talk about with regard did we want to look at the did you want us to review the presentation? I wasn't sure.

We can absolutely if you want. Mia or Ms. Vang, I just want to say thank you so much for sharing this with me. Ms. Vang converted the original PDF form that former community advisor, Hornsworth, had and brought it back into a PowerPoint format that could then use as a deck slide that can be edited.

So thank you. Because I had otherwise, I would have had to recreate that probably all from scratch because I am not that tech savvy. I am a little bit incompetent in terms of technology despite my age. So I get made fun of all the time by my students actually for that. So but I will probably are there any questions regarding the presentation as it stands currently or anything that council members or community advisers had for me regarding any changes or updates?

So otherwise, I will probably come back at a future at our next meeting with some suggested changes to this. Mhmm. Otherwise, the bulk of the the bones of this are not changing. A little bit of flesh might change a little bit here and there, but the bones of this are fantastic and that's kudos to mister Farnsworth for that. I'm sad to see that he's leaving.

My trying to think here. I didn't see anything on here that really warranted my asking questions or anything or that anything of that sort. Was there anything that any of you had questions about regarding this at this point in time?

Just got a question on slide five where it says the first bullet nineteen ninety's audits. I just don't know what that's regarding. I'm

sorry, I didn't mean to interrupt you.

No, yeah, it's just I don't know that piece of the history.

So I need to follow-up back with former chair of this committee, council former council member Prince on that actually, because I know that she has a lot of institutional knowledge regarding how this committee was first. How it came into being, how it was created. Because there used to be from my understanding, my conversations with her, and Madam Chair, please correct me if I am wrong, that during the nineties there was a legislative audit program here in the council that was part of council research. And it was not a large team, but they did quite a bit of work over a period of time. And then there were budget cuts that basically came through from a particular mayor.

And so that were pushed pretty hard and that whole program went away. And so in 2018, former council member Prince and chair Necker, they pretty much led the recreation of an of a basically legislative audit activity here at City Hall. So and I love it. I love it. But is that is do I have the gist of the history correct?

Yeah. That's exactly right. And I think what this prints could provide you with is some of the specific examples of the audits they did in the nineties, which were just as juicy as the ones that we do today. Believe, you know, Miss Vang?

Yes. I just want to share that those audits that were previously conducted in the 1990s are uploaded to our audit committee website And for historical they were indeed done in house because at the time council was a full time council and a full staff. So they were able to do have that level capacity to do that level study. Yeah.

I've read some of those audit reports. They're delightful. Yeah.

Any other comments on the presentation itself? The only thing I would suggest is just that I think as you're talking about the former, the audits that we've completed, just to give folks a sense of what came out of those, like what maybe some of the specific recommendations were and especially ones that were actually followed up on. You know, we talk often here about like, did the study, what actually changed? And I think there are concrete examples of how departments have changed things, including by the way, just so everyone knows, Councilmember Coleman and I were at the CIV committee meeting last Monday and they referenced we were there because the city council was talking about their work as part of our budget process and have been more interested in learning more about their work as part of our budget process and also because of our audit. They welcomed us to their committee meeting and while we were there they talked.

They actually had a report not because we were there, just part of our normal agenda from the commission member who is going to be leading the work on taking our recommendations forward. So without us, no influence from us, that's just happening on its own, and I just thought people would like to know that. So having some examples of, like, what actually changes on the ground because of these audits, I think, is maybe a good addition to the presentation.

Duly noted.

I'm going to move us along just because unless there's any other comments on this, because I know we have just twelve minutes left. I think, for the sake of time, topic selection process, I'm going just hold on because I think we've got it written down, we know it. I don't know if there's lot to discuss there.

No, I think it's really related to what community advisor McVeigh just talked. And this is just kind of a broad outline of our process. And it doesn't deviate too much from the way we've always done it, but it's outlined here.

Great.

Excellent. And that's a good piece to share when you're presenting to these district councils as well. Okay. Great. So then with that, we do have we need to talk about our community advisor appointment timeline. I don't know if it was explicitly said, but Mr. Friendsworth unfortunately has moved across the river. Yes. So he is still alive and well and wonderful, but legally

can no longer be part

of his Resident.

Yes. He can be a Minneapolis Audit Committee member if you would like, but he cannot be with us. So we do have an opening. Or Ms. Vang, sorry. Do you want to walk us through the timeline? Yep.

Thank you, Chair Nager. I just wanted to kind of walk you through this timeline. It's very similar to the last timeline that we used to seat Community Advisor McVeigh to the audit committee. So it's really quick. It's a two week posting.

And then we would then have two weeks to kind of review, rank our choices, and then invite our top three candidates for an interview or top two. And then we go through the process of making our final selection and proposing that to the city council to vote on the matter. And then hopefully we can see the new member in January. But given that the holiday, and I wasn't sure if the committee would contemplate extending the two weeks to maybe possibly four weeks, so that it brings us into the new year. Given that Thanksgiving, Christmas, and possibly New Year are in the mix.

We may not have as much of as many qualified individuals as we would normally hope for during the normal season. So, this is it. What you see for you is the application. It doesn't change much from the last time. We kept the question very similar from the last time when we had a vacancy. If you wish to change those questions now, we can possibly change it as well. I thought they were very simple. We did away with the resume and cover letter, but we wanted to give folks the option if they wanted to submit that for the audit committee to consider. And not everybody takes advantage of that and that's fine because we eliminated that part. And then, a press release will be ready to go out tomorrow.

The proposal is that this goes out tomorrow. And if you would like to submit a quote for us to release with the press release, I would certainly welcome that as well. But the goal is that this application will be live tomorrow and that ends at December 3, so that I would have enough time to compile the applications and then provide that information to the committee to review and rank. So we're working within a very short timeframe.

I would just off the bat suggest that we do have a longer posting. I think two weeks with the holidays is not probably enough. So I think it's alright if we have a little bit more time before we have the seat filled, but I I would suggest a four week maybe, miss Coleman.

Yeah. Thank you. I'm just trying to think this through because in an ideal world, I feel like we would have the first two weeks in January to have people apply. Obviously, that's much much longer, but, you know, just thinking about when people are thinking about taking on new commitments in the cycle of the year. But at a minimum, yeah, I would certainly support giving the posting open for longer. I think having it open primarily over Thanksgiving week is just will not get us the the millions of applicants that I trust we would get if we posted it for a little bit longer. Okay.

And then ideally, I think if we could line up interviews with the existing meeting and maybe we just have to plan for a little bit of a longer meeting that day. I think that would be better than scheduling a different time for us all to come back together just to do interviews. Looking to my colleagues on to see if I'm right about that or not. Could also consider having a meeting in February. Well, depending on I guess it depends on where things land after you do the four week extension or four week posting. I guess that would be moving us towards the January to have somebody seated, right? Right. If we did four weeks?

So we would. And because we wouldn't have a February meeting unless the committee deem it necessary, then we won't have an interview until March.

So maybe we should plan on the sorry, let's go on on the interview on January 20 at

our meeting then. That I would let me pull up my calendar. Give me a second.

Maureen's. I was just going to ask, the interview have to take place at

a meeting? Okay. Yeah. Unless the committee would like to meet in a special meeting, then we'll find some additional time. But if we go four weeks, then it's due Christmas Eve, the application. I will stop you there. Unfortunately, I won't be working on Christmas Eve. And then if we extend for five weeks, it brings us to New Year's Eve. So then, I don't know if you would like it to extend for seven weeks, so that it brings the application would then be due at January 7. We have a meeting on the twentieth.

Would that give you enough time between the seventh and the ideally, would like a week to contact all the potential candidate of when we will meet. And usually, that takes a little bit of finagling. Ms. Cole?

Would we thank you. Do would we be able to have access to the applications on a rolling basis?

To be quite honest, you should you can, but we normally don't get a slew of application to the very end. Unfortunately, that's how it's worked here. The pattern hasn't changed too much. Yeah. That makes sense. And we would need, obviously, a resolution that goes to counsel to vote on this new member before he or she can be seated. I think my recommendation would be that

we make it due on the fifth, January 5. And then we have two weeks and a day before we would need to meet and interview the person, the people on January 20. And I think if we're all on our game and we usually don't get millions of applications as we know, we get a few, we can look at them. We often, I think, interview most of the people who apply. So it's not like a hard decision as to who we invite. And I think miss Vang, we've done this in the past where we've told people on the application, this is when you should be prepared to interview so they know January 20 at four or whatever. And then we could have the person seated by the March meeting. Okay. That'd be fine. Yeah. I'm What's that?

If the committee feels that that that's an appropriate timeline, I will certainly make the changes. How

do folks feel about doing sort of a I'm not this may not be a good idea, but I'm thinking we're we're all gonna be promoting the topic selection survey and we also wanna be promoting the fact that we have an opening. Does it make sense to do sort of a combined audit committee posting, you know, hey, everyone. Two great opportunities. One, to tell us your

ideas and two, to join us.

Like, it's faster. I'm just trying to think through because I don't otherwise, we're doing two postings at the same time about the same thing sort of, and maybe we should draft a blurb that kind of incorporates both. Okay. What what do others think?

I would support that. They might actually garner more attention, actually. The way people's brains are wired, having that out there. It sounds a little counterintuitive, but there is some research that suggests that when you have two when you have a when you release information as like a governmental entity and you have two pieces of like two story lines simultaneously commingling with one another, it actually can garner more attention because there's just more information being presented basically.

That is counterintuitive but fascinating.

It's a lizard brain thing.

Thank you. Okay. Well, let's do it if there's no objections. So we'll look for a revised blurb for both topic selections and this process, the thing. And then I think maybe we could wait a wait to post until I mean, we could give people some time to submit quotes, for example, maybe do it at the end of this week instead of tomorrow just because now we're giving people

Sure.

A lot

more time to apply.

Sure. I would okay. So then ideally, Friday would be my my due date to or would you like next Monday We do

next Monday and we get people till the end of the week.

Okay. Great.

Okay. All right. Anything else to come before us? Looks like we're at the end of our formal agenda. Anything else you need from us, Ms. Vang? All right. Well, thank you all for a productive meeting.

We got

a ton done as always. I hope everyone has a wonderful Thanksgiving. It's only next to each other and we are adjourned.