Where AI and Machine Learning can Assist Public Works (and Where it Falls Short)

0:00:00.1 Bailey Dickman: To advance, streamline and modernize their water utility, Sugar Land, Texas built what they call a dual horizon predictive management system. This dual predictive model works to predict long-term pipe wear and flags pipes at risk using machine learning. Just last week, Alence Poudel, the senior engineering manager in Sugar Land, presented on these findings at PWX in a session about AI, engineering, and future-proofing Sugar Land’s award-winning water system. We caught up with him in Houston to learn more.

0:00:30.3 Alence Poudel: The dual horizon model works by looking at two different things. It’s trying to solve two major problems that every local government has going on: a capital problem for long-term risk reduction and operational problem for immediate impact that’s gonna happen tomorrow, next summer, and so on.

0:00:48.6 Bailey Dickman: And that’s the long story short, the dual horizon model works. And because Alence is a PhD student who publishes a lot of his research in peer-reviewed publications, we know that this model has flagged more than 82% of the pipes that failed in Sugar Land. By 2025, main breaks had dropped by 25%. That’s really, really astounding numbers. But there’s something equally important to note here. AI hype is reaching wildly unmanageable levels, and sometimes it falls short.

0:01:19.3 Alence Poudel: With the AI models, the problem was the worse the model was performing, the [more] confident it was. So one of the models said that you need to do this over that definitively. It even cited three papers. When we tested that, as we feared, more than 50% of the recommendation by the AI models did not exist. Those were the hallucinated output.

0:01:43.1 Bailey Dickman: We know everyone is excited to introduce tech to meet the demands of growing costs, budgetary concerns, and community expectations. But both Alence and his colleague Trevor Surface, the utility field operations manager in Sugar Land, who also presented, agree that introducing a shiny new piece of sophisticated tech might not always be smart. Here’s Trevor with more.

0:02:03.4 Trevor Surface: Instead of just going out and trying to use the flashy new buzzwords, municipalities really need to ask themselves, “Are we ultimately gonna benefit [from] this? Is there gonna be staff time? Or are we gonna solve an issue better than how we’re doing it before?” Right? Because we ran the same modeling with our wastewater gravity line, and we found that our old way of doing it was better. It warranted better results because we didn’t have the data inputs.

0:02:30.5 Bailey Dickman: The promise of what this technology can do is genuinely incredible. But let’s not forget that we still have a long way to go.

0:02:36.8 Alence Poudel: The conclusion behind this was you really need to audit your output from the large language model against your tried and tested model. You benchmark something before you decide to just blindly go after the result that’s given to you by the AI models. Not saying that the AI models are not great, they’re amazing, but for the infrastructure decision-making, they are not there yet.

0:02:57.7 Bailey Dickman: And of course, technology will never be able to replace human judgment.

0:03:02.3 Alence Poudel: That’s why the goal behind this was, you look at this, now we are telling you empirically that still the AI models are not at that level to completely replace the human judgment.

0:03:12.9 Bailey Dickman: Both Trevor, an operations manager, and Alence, an engineer with a doctorate in his future, agree wholeheartedly that if the engineering department and those in the field aren’t on the same page, none of this is going to work.

0:03:23.9 Alence Poudel: And that’s the biggest misalignment that we see. I would say first, talk to the field guys, explain to them what you are trying to do.

0:03:31.8 Trevor Surface: Look, engineers talk in big fancy language that you may not understand at first, but at the end of the day, they may not have gotten the exposure in the field. So invite them out to a main break, invite them out to a road repair, invite them out to some different scenarios and let them see what it takes to actually do the repairs and really start to bridge that gap that I think exists between engineers and field employees.

0:03:55.0 Bailey Dickman: Fresh off of our visit in PWX where these interviews were recorded on-site, Public Works Radio is back. And as you can probably tell, we have a lot to cover today. Let’s get into it.

0:04:04.9 Bailey Dickman: Correct me if I’m wrong. Effectively, there are a bunch of different ways that you can, different probability curves that you can use to estimate water main pipe length of life. You can use, the manufacturers have a very optimistic curve of how long their materials will last, which is fair because they have to sell it, they build it in a lab, they test it. But then if you just model just using break history, obviously that’s a very pessimistic line. It’s very, obviously it’s only the bad stuff gets put in there. You don’t see the good stuff that gets put in. Can you talk to me about this anchored Weibull, this enhanced model that you use to look at both the length of the pipe, how long it’s been in the ground, what material it is, has it been broken before? Can you talk to me just a little bit more about that, using that to develop that risk model process for Sugar Land?

0:05:00.9 Alence Poudel: Yes. So the dual horizon model works by looking at two different things. It’s trying to solve two major problems that every local government has going on, a capital problem for long-term risk reduction and operational problem for immediate impact that’s gonna happen tomorrow, next summer, and so on. What we found out that manufacturer curve is an ideal starting point because they manufacture the product, they will tell us how it will look like and all. But when it’s installed in the ground, that’s outside their control because it really depends. If the pipe was not installed properly, it’s not manufacturer’s problem. It’s what, contractor…

0:05:20.4 Bailey Dickman: The installation. Yeah.

0:05:40.0 Alence Poudel: And then depending on the ground condition, on the depth of the water table, the soil condition, the other factors. Was there another construction nearby? Or maybe you’re trying to install another line and you hit the line, the existing water line by mistake. Now, that cannot be taken against the manufacturer. That’s the existing soil condition or the external factors. But the challenge is you can also not just take what happened in the past and say, “Well, my line was supposed to last for 70 … 75 years as per the manufacturer, but in reality it’s only working good for 40 years, so I should model it saying 40 years.” No, because those are a sample in your entire population. And if you start replacing those lines at year 40 or so, the problem is then you’re replacing a lot of lines way too earlier than you should be. And the result is going to be, while you’re going to replace the mains that’s newer, you’re going to start experiencing break somewhere else. And now you don’t have an answer because you literally went and replaced something. How do you fix it?

0:06:46.9 Alence Poudel: So we came with this idea of anchoring these two distributions into one. We use Weibull because we did some statistical testing and validation to see what probabilistic model or what distribution would best fit this. And it’s in the paper on how we did it, why we did it, and what are the metrics to validate the hypothesis on, like your p-value, your KS test, your AD test. We did all of that to find out that Weibull in our condition, I’m not saying in general, in our condition, Weibull was the better fitting distribution compared to others. Anchoring because we also wanted to take what was manufacturer showing, what is our data showing, what is our local break information showing. And then when we put that anchored curve into a pattern and then started testing what will happen or what happened in the past, it showed a close overlap of what we’re experiencing in the city. That’s why we call it an anchored Weibull distribution, because that was a better representation of the actual condition in the city of Sugar Land. Now, to solve the operations problem, what we did was we took that and we ran a machine learning algorithm using gradient boosting approach and added the break history worth 25-plus years. Then we also added the joint type because there is so many inconsistency on how we write information. So we made it more consistent, normalized everything, added what happened in the past, and then tested several hypotheses to find out, okay, what are the factors that’s actually causing a break in our city?

0:08:20.0 Alence Poudel: Where do we need to put more emphasis on? Because a model could show 20 different reasons for a break. So is it possible to capture everything? Yes, but do we have enough time? Maybe not. So then it also helps us be more strategic on what are the most important factors that we would need to look at that needs to be captured and what factors could be taken care of later when we have time availability and so forth. And when you combine both this anchored Weibull and machine learning approach and then cluster this into not projects but program, that starts giving you an actual CIP program which would not only work towards solving your short-term problem, but also focus on the long-term risk reduction. And as a result, you’ll be investing in your CIP and hopefully your operation emergency cost dollars is going to start slowly going down because now you’re solving immediate problem as well as long-term problem.

0:09:15.3 Bailey Dickman: As well as future problems. Yeah.

0:09:18.4 Bailey Dickman: We talked today, you guys had a session about AI and machine learning in your pipes and how you made a plan that actually could be implemented with that in Sugar Land. Do you want to just talk a little bit about that?

0:09:31.4 Trevor Surface: Yeah, I think that’s a great question. So we have been kind of a joint venture between our utilities and engineering department working through applying machine learning with our long-term CIP program for a while. So instead of the old way, which I think I referenced in the presentation, taking a map and throwing a dartboard to say where we’re going to rehab pipes, it turned into thinking about it a little bit smarter. We already have a good work order history, we already have the GIS data, includes material, age, diameter. But why don’t we look at it through the history of breaks through the various geographic locations of the city and start to run newly kind of hot topic machine learning into how we go with our CIP? Our five-year CIP, that was great, very successful. I think we’ve been running that for three-ish years. About two or three years ago, the question came up of, “Hey, this five-year CIP is great, but what about next year? Why don’t we take the long-term and see how does it run short-term?” That not only helps us determine where, if we have extra funding on CIP, could we get maybe a smaller section to rehab? But for our O&M side of the house, where can we start to maybe proactively do leak detection, acoustic leak detection, or visual inspections? But then also if I have a main break, can we reference a modeled map that tells us is it beneficial to just do a temporary clamp or should I splice in a whole 20-foot segment that’s going to prevent me from going back out there disturbing dirt and disturbing residents? That’s kind of what led us to the overall dual horizon planning system that we run now.

0:11:26.8 Bailey Dickman: When it comes to that on-the-spot leak detection, you mentioned in your presentation that some of your field crews could predict where leaks would happen. Talk a little bit more about that just for maybe engineers who don’t often work directly with field crews or some of our other podcast listeners.

0:11:44.3 Trevor Surface: Yeah, for sure. So we’ve had employees in the city of Sugar Land Utilities Department been there 36 years. Some folks in our groundwater have been there 50 years. They often joke that it was a lot easier when they first started, but there was also five houses that they had to deal with. But no. So we ran into the problem of we have all this knowledge, right? We have all this knowledge is data. Data is knowledge. How do we start to quantify the qualitative data in our more tenured employees?

0:12:20.8 Trevor Surface: And the push there is that we could, there’s a few different options. We could have them circle a map and then transpose that digitally. We could have them interview them and ask them key questions. But then we thought, well, we have 20 years, well, almost 30 years of work order history data that these notes that they’ve read, QA/QC’d, they’ve been putting these notes into our work order management system. That is what turned into the field notes kind of data entry into the models that we’re talking about today. And that’s where we’re starting to use some of the fieldwork, some of the knowledge that we have there into what we do. The reason that it’s in, so our work order management system just kind of gets all of that free text data into one spot. So it’s a lot easier to pull than say an Excel spreadsheet, Word document, handwritten notes. But it then allows that if a supervisor reads a work order off of a main break that happened yesterday, well, they have the ability to go in and put their notes and say, “This is a hot topic area,” and we can flag it in our model if we needed to.

0:13:33.1 Bailey Dickman: What conversations should we be having about AI in water resources?

0:13:38.4 Alence Poudel: Oh, that’s a big question. And I typically start with it’s not the model, it’s the end goal. I have seen and based on my limited experience, a lot of times us engineers are too fixated on the model. And to an extent with the, I call it, I don’t know if that’s the exact thing or if it’s my creation, but 2023, I call the ChatGPT boom. And after that, every engineer wants to be an engineer with a machine learning expertise and the LLM model developer or something because it’s like a race. Everybody wants to do, “This is my best and all.” And I think that’s the starting problem because it’s not about the model. It’s about the value that we’re trying to create. For utility, the challenge is we want to ensure that with the limited money, we do the best we can.

0:14:31.5 Alence Poudel: If we are planning for the CIP projects, capital improvement, it’s more about, okay, with the available dollars that the city has, $2 million, $5 million depending on your city’s size, what are the projects that you need to do in order to ensure the risk level in your system? And then when you have that answer, then you start using the tools. Do you want to go with a simple linear old-age approach, or do you want to add some statistics to it and then make it a probabilistic model? Or do you want to further add the layer of machine learning to make it more of advanced algorithms? Or do you also want to use AI to make it even advanced model? So it really depends, and I hope that our engineers think it that way as well on first identify the problem. And these tools, AI, machine learning, statistics and all, these are the mechanisms to get you towards solving the problem. This shouldn’t be the whole thing or whole focus on your modeling approach. In our case, we did the same thing for our water mains. We started with linear approach, then we tested that against a probabilistic modeling. Then we tested that against the machine learning model to see, let’s first establish the benchmark and let’s compare the output against that established benchmark.

0:15:49.9 Bailey Dickman: What conversation should we be having when we try and merge these O&M, these predictive on-the-fly fixes with some of these more long-term effects? What’s the major talking point that you want more people to be thinking about?

0:16:09.0 Trevor Surface: I think the biggest question that any municipality needs to ask is what are they trying to solve? What’s the problem? What are we trying to solve? Are we data ready, and what’s the need? And I think that’s gonna drill down whether people just want to use the new flashy topic, or is there an old-school method of doing it? We’re not here to push everybody needs to be machine learning, right? At the end of the day, we’re just showing what works for Sugar Land. Instead of just going out and trying to use the flashy new buzzwords, municipalities really need to ask themselves, “Are we ultimately gonna benefit (from) this? Is there gonna be staff time, or are we gonna solve an issue better than how we’re doing it before?” Right? Because we ran the same modeling with our wastewater gravity line, and we found that our old way of doing it was better. It warranted better results because we didn’t have the data inputs. And I think that with that in mind, we’re more informed on how to proceed with the different assets the city runs, right? Like, could we run the same idea with our street network? Could we run it with our drainage network? I think that we now have an idea of all the inputs and data inputs necessary to say whether we need to go that way or not.

0:17:26.9 Bailey Dickman: What chance does AI even have to compete against a human brain, seven human brains working together to battle out one solution for a complex engineering problem?

0:17:39.0 Alence Poudel: That’s a great question. So because of that issue we faced, the way we structured is that our first part was auditing against a local risk model. The second one was we wanted to test man versus the machine, where we had a group of 20 infrastructure professionals. We asked them identical questions on three different realistic scenarios on critical infrastructure decision-making, capital planning, and emergency response. And what would you do? And we made not the exact case, but more of a hypothetical scenario to add more emphasis on things on, you have $500,000.

0:17:59.0 Bailey Dickman: Yeah.

0:18:18.9 Alence Poudel: You have, let’s take an example, you have two assets that really need replacement, and both are in the worst possible condition. But with the limited budget, what would you do and why? And describe the reason and how confident are you in your answers. We reached out to 20 different infrastructure professionals ranging experience from entry-level to 25-plus years’ experience. Local government, consulting, academia, everybody were involved because we wanted to have that diversified pool. And then we tested six commercial generative AI models because a lot of people use that, and we wanted to see whether or not the decisions that’s being recommended by AI model align or not with the humans. We did not keep the questions open-ended because then it could go all directions. So we made it structured, exact same questions, exact same prompt to humans and to the AI models.

0:19:12.8 Alence Poudel: What we found out was both senior experts and junior experts, so the way we describe is anybody with 10 years or more of experience is senior, anybody with 10 or low is junior because we wanted to build that cohort, both our senior experts and junior experts performed really well than any of the AI models, to a point that there was something very interesting. We did some statistical testing, and on an average, our humans said they are three or two and a half scale or point confident on a scale of one to five on their response. And the reasoning was because those are the problems we really need to solve, there is not a right answer. It really depends on the preference, and it really depends on the factors like compliance, your standards, the next year budget, the cascading impact, the leadership decisions, the strategic plans and all, which may not be what we are looking for, but that’s the realistic answer. There is no right answer in public works. It’s always gonna be depending on, are you meeting the compliance requirement? Are you meeting your standard? Are you meeting your city’s strategic goal and vision? You have to do a project, and you have to hope that the other line doesn’t break, and eventually you have more money, you can go and do that. Because ideally, what do we want? Infinite pool of money so that we can go and replace everything and be safe.

0:20:37.0 Alence Poudel: With the AI models, the problem was the worst the model was performing, the (more) confident it was. So one of the models said that you need to do this over that definitively. It even cited three papers and a report saying as per the research by following authors and this national report, you would need to emphasize this over that. And I am four and a half confident out of five. When we tested that, as we feared, more than 50% of the recommendation by the AI models did not exist. Those were the hallucinated output. And that was the goal behind this project. Because for an entry-level engineer or somebody early in their career, if you were to blindly look at that, when I first look at the reference, the formatting, the journal’s name, the page number, even the link…

0:21:25.0 Bailey Dickman: It all looks legit.

0:21:25.8 Alence Poudel: It looks so legit that I am glad I tried to open that link to find out it doesn’t exist. To a point, there is a paper that they said, as per the report from this audit, you need to do following things. When I opened and checked, that author didn’t even work in this field. The author was in the field of biology. And that’s the problem, and that was the goal we wanted to show not just to local government, but to everybody, consulting engineers, because they are tied with us as well, academia, students who wants to work in this field, that you really need to go back to auditing and reviewing the answers. You cannot blindly trust it because we asked the exact same questions first to local risk model and show the result. Then we asked it to infrastructure professional to see, can we replace human judgment? We found out that no.

0:22:15.5 Bailey Dickman: Absolutely not.

0:22:17.0 Alence Poudel: That’s why the goal behind this was, you look at this, now we are telling you empirically that still the AI models are not at that level to completely replace the human judgment.

0:22:27.3 Bailey Dickman: What’s been one thing that surprised you while working with this machine learning model about what’s underneath your feet? What misconceptions did you have about your data, your assets before you got into this?

0:22:39.7 Trevor Surface: I think the biggest thing that we really learned is just how… This is gonna sound very weird, but how accurate everything you hear from the more tenured employees is actually true. And I say that in jest, but I think that at the end of the day, we could sit here with an employee that’s been at the city for decades, they’re gonna tell you everything that they’ve been, that they’ve done. And with this, you start to really see those hotspots visualized. And I think that it’s really kind of started to better understand just how much knowledge is there and how to tap it in an appropriate way.

0:23:26.7 Bailey Dickman: How can AI be really, really helpful for small and mid-sized cities? What’s the use case for a smaller town who maybe has less resources than Sugar Land to even begin to use something like this?

0:23:43.2 Alence Poudel: So I would say this way, AI is a great tool. It really helps to be more efficient. For example, we could take a couple of days to come with an idea. Now, it would help us fine-tune it and then get it done in 30 minutes or an hour. That doesn’t mean that AI is doing the work. The way I see AI being very helpful is using AI as a first pass or building that original checklist. You’re working on an idea, you have bunch of written brainstorming notes and everything in there, and you organize everything in there and then you start doing your research first; you get your checklist, then you as the expert, the human engineer involved or utility professionals, would use that checklist to audit it, to verify what’s needed, what’s not, how can you improve it, and then you add more feedback to it. Then what would an AI do? It would start refining based on what you have described now. Now you don’t have a vague problem, you have a research problem, you started working on it, you got your checklist, you reviewed it, you added feedback, and then you put that feedback back into the AI model to refine it.

0:24:48.6 Alence Poudel: And this iterative process goes on and on where you also communicate with other stakeholders. For example, an engineer in CIP planning team would talk with the city engineer, would talk with the assistant city engineer, the field operations manager, the utility director to identify the problem. And you keep refining the output given by AI, but through human intervention, human input in it, and ultimately, the result is going to be, or hopefully it becomes, a very efficient product that you are actually trying to take forward. Not just completely built by AI, not just completely worked on by human, but the nuances that human will add and then refined by AI. I think that would be the best possible use case of AI in public works or local government.

0:25:33.6 Bailey Dickman: What advice do you have for engineers in sort of getting buy-in or talking to the field operations folks in a way that is helpful for everybody?

0:25:43.8 Alence Poudel: That’s a great question. I do see a lot of engineers, they don’t go to the field, which is very interesting because you need to go to the field and you need to talk to the field guys. If you’re trying to build any kind of risk model or planning model and all, you need to communicate with them to ask them what are the problems that they are seeing in the field. And then it comes out to be whether or not your model is actually showing that output or not. Because your model could tell you a story, field guys are experiencing a different problem, and that’s the biggest misalignment that we see more often than not. So for engineers, I would say first, talk to the field guys. Explain to them what you are trying to do. Get feedback from them. Ask them, you don’t need to ask them, “Oh, can you tell me if my probabilistic equation is correct or not?” No, that’s the wrong question. You need to ask them, “Okay, are you experiencing a water main break in this location?” They’ll say yes or no. Or they’ll be like, “I’m just experiencing a leak.” Or they would say, “Yes, but it was due to poor construction and all.” But you would have all the insights on what they are experiencing. Then you can ask them, “What are the other areas that you’re seeing the problem?” Then you go back, you run your model, and then you plan out, you map out your output, and then see the insights that you got from your field team, your operations team. Are you able to see it in your model?

0:27:03.0 Alence Poudel: And if not, then you need to have another communication saying, “Hey, my model is showing something else. Can you please verify it?” Because if you make your operations team an integral part of your decision-making, your program is likely to succeed because they will provide their feedback. You see an issue in your model, you ask them, and they can tell you, “Yes, we saw that, but it was two years ago.” So then the result is your model is working fine, but your data is incorrect. Without talking to field, you’d not find that information. Or sometimes they would be like, “That is correct, but we have another problem that we need to take care of.” So then you need to reprioritize your model to fix what’s seen in the field. Because you can run any fancy model, you can come with the budget numbers and other stuff, but what happens in the field is the first thing you need to go and tackle it. And then the part that I recommend all the engineers is if you are working in the predictive modeling and all, please know your statistics. Because you can only validate your model if you know the background statistics behind it. Do not blindly rely on AI models to translate statistical output for you. You would need to know what you are doing, why are you doing it, how would you test it, how would you validate it. Then you can use AI model to make it faster and all, but know the fundamentals, know the theory behind it, and know why you are doing it.

0:28:28.1 Bailey Dickman: Is there anything else that you want to tell to somebody who’s maybe a field crew supervisor or somebody who works more on the outside in the crew side about how useful this kind of thing can be for them?

0:28:40.9 Trevor Surface: Yeah, don’t be afraid to talk to engineers. Look, engineers talk in big fancy language that you may not understand at first, but at the end of the day, everything that they learned in their books and classrooms they may not have gotten the exposure in the field. So invite them out to a main break, invite them out to a road repair, invite them out to some different scenarios and let them see what it takes to actually do the repairs and really start to bridge that gap that I think exists between engineers and field employees. Because there’s a lot of knowledge and a lot of the same work that we’re doing that would be improved with that cross-training, that cross-learning. Not only external networking and figuring out how other cities are doing XYZ or learning how cool new products vendors are doing, but even internal networking is very important. And I think that really opens a lot of eyes and it allows people to understand the work that goes into what engineers see on the plans, but then vice versa, it allows the field staff to understand the work that goes into engineering a new water pipe or engineering a new subdivision. They may not fully understand all the work and regulations and processes that go into that.

0:29:57.0 Bailey Dickman: Thank you for listening to Public Works Radio, the official voice of the American Public Works Association. And thank you to our guests, Alence Poudel and Trevor Surface, for finding time to sit down with us in Houston. We have links to the published studies that we referenced available in our show notes, so be sure to check those out. And hey, be sure to subscribe, rate, and review this show. Forward us along to a friend, and don’t be shy about dropping us a note over at podcast@apwa.org so we can hear your feedback directly. We’ll catch you next time.