Define Like a Butterfly, Defend Like a Bee with Jason Skow

Rhett Really excited to be with you guys on this PPIM edition of pipeline things our first guest I have to say is Magnificent. And Chris I grew up, you know parts of the show I played Mike Tyson's punch out and there used to be a character in there that was dance like a butterfly sting like a bee that was what I was thinking. Chris This episode actually kind of makes me think of a little bit differently with 607 maybe it's more around defined like a butterfly and defend like a bee. Rhett Audience judge for yourself as you join us with Jason Scow. *Intro* Rhett Welcome to the PPIM edition of Pipeline Things. I am your host, Rhett Dotson, my esteemed co -host, Christopher Deleon, and our even more esteemed, second time, repeat guest, Jason Skow. Chris Couldn’t to keep him away. Rhett Jason, welcome. Jason Happy to be here, gentlemen. Rhett So, to our audience, you are probably aware of it now that we are going to be covering papers at PPIM, and this is one of the papers that we're chosen. Really excited to have you on. But before we get too deep, we always like to start with segue, discussion of things. You have kids, right? Jason I do have kids. Rhett Do they play Roblox by any chance? Jason No, they don't play Roblox. Rhett That's a shame. Chris, do your kids play Roblox? Chris Never. Rhett Okay, well, my kids have gotten into Roblox. Chris Did you just fail? Rhett No, I didn't. The segue way is still valid. Jason Okay, somebody plays Roblox here. Rhett Somebody, it's not me. I will tell you it's not me, but it is my kids. And so there is this sub game within Roblox called Dress to Impress. And as part of Dress to Impress, like you go through and they give you a genre and then you have to go pick out your clothes and they do like a mock fashion show. So I have three girls. Jason This sounds like a different game than what I was thinking of. Rhett You're like wondering like, how does this relate to the pipeline industry or me or anything that's going on? Jason I have 100 % faith that you can draw that line. Chris Bring this home, baby. We're all curious. Rhett There are superpowers that as a host of a podcast you have, and this is one of them, the ability to draw other parallels from otherwise completely unrelated things. So my three daughters, Reece, Aubrey, and Peyton, love to play this game right now. They've even drugged my wife into playing this game, and they will, and it's really, really humorous to see them do this. Jason And it's Say Yes to the Dress, Roblox Edition. Rhett Say Yes to the Dress rhymes, but no, it's Dress to Impress. Jason Oh, sorry. Rhett Right, And you should know, this game's permeated just enough for my son had his band homecoming and the theme was dressed to impress. And there was mass confusion at the high school level, whether it meant dressed to impress like Roblox or what you would conventionally think of, which is dress up for homecoming. Where I'm going with this is, you clearly came dressed to impress. I want the audience to know, if you're not watching on the YouTube, this is your moment, I thought we were shooting usually when we shoot virtual, I'm relaxed attire. I wasn't thinking PPIM attire. So I'm here in my 2022 Astros hoodie. Chris showed up with Chris On brand hoodie. Rhett Is it? I mean, it's not Chris It's on brand. Rhett I don't, it's green. It's got that. Chris What is Sarah doing? Am I on brand? Rhett Point is, Jason, thanks for dressing up like you were at a conference to make it look Chris Like if this is PPIM. Rhett Yes, 'cause we did shoot this episode early, so if y 'all are wondering, we don't have the full PPIM environment here. We had to get Jason's guy while he was here, which is shortly before PPIM. Chris We had to have Jason back on, and so we made it happen. Rhett But we were like, I look like I'm dressed to depress. Right? You came here- Jason Thank you. Yeah, well, I mean- Rhett So what do you think about the link? Does it make sense? Jason Yeah, the link makes sense. And I'm a little overdressed, I would say, my theory is, you know, you fake it till you make it. Okay, so you look at the little scene, you guys are running the show here. You got all this, you know, industry recognition authority. If you will. So I got to dress up here just to keep it- Rhett Just know if I put the hood on during the show, does it lessen? Chris Oh, this is a dark. Jason You look a little more dangerous. Yeah, I don't know what you're going for. Rhett All right, so audience we're fully gauged here y 'all are with us now. Hopefully you enjoy our banter back and forth so today's topic the paper I'm gonna go right into it So we switch gears from dress to impress and that's high-level segue. Sure. Can you do it? Can you make that gear shift that way? Jason I'm gonna keep talking about dress wear, but go ahead— Chris I'm curious if he's about to parallel 607 requirements with fake it till you make it. Rhett There are limits Jason There's a link there, there's a link there. Rhett We're gonna explore it, Ms. Producer. If you will please write that down. That question will be coming up in the second half of the episode. What is the link between 607 and Fake It 'til You Make It? So, but your paper was estimating ILI tool performance and identifying unique pipe populations for material verification. Jason That's a long title. Rhett It is, I was gonna say, and I usually target funny titles or that is all business. Jason That title is pretty much half the paper. Rhett That's it. That's all we need to say about it. Jason You know, I can dive into that title a little bit more, but sure. Rhett Alright, so phone a friend, phone a friend, Chris. Chris It's so relevant. It's so relevant, right? Rhett So, give us the relevance, right? Chris So, if we go back and think of RIN 1, obviously, mega rule. So, we have material properties verification 192607 and in it, it's basically if you don't have TVC material records, you have to go get them. So how do I do that? And the meat of it that we often find is in, I think it's paragraph E. So 607 E. And I think basically what we're trying to accomplish is one of the first things you do is you establish populations. Once you establish populations, and you go sample those populations to identify what it's made out of. And if you find something that's not what you thought it to be, you have to do an alternate sampling approach. And guess what we don't want to end up in? Rhett Alternative sampling. Chris We don't want to end up in all, well, I mean, ideally you don't, right? So you want to establish populations. And this paper is going to help us understand a little bit how we can do that with ILI. Rhett And those populations were defined on specific characteristics, right? Chris 100%, right Rhett You know, is it seven characteristics? Jason Yeah, you're just going to throw in the word populations. Are we talking about a bunch of people here, Chris? Come on, what are we talking about? Chris There is a numerical definition of population that we're gonna talk about today. And obviously, I mean, we're pipeline things, but obviously we have ILI gravity, right? So normally when we do things, we normally gravitate towards how you can use in-line inspection. And fortunately, we also have references, and we know that ILI can be a big help, right? It produces a lot of discrete data that we can then use to make better decisions. Hopefully not end up in alternate sampling plans because you found something you didn't know. Rhett So Jason, now that Chris has set the stage. We dropped that voice down, make it deep. So now that Chris has set the stage, what's going on in your paper? What's the driver for this paper? And I'm actually genuinely curious, because I feel like 607 was talked about a lot, maybe in the last two or three years, and it doesn't feel like it's been the same level of Maybe some stuff around alternative sampling, but what brought you to this paper? What's the challenge? Jason Yeah, the challenge in this paper is You know all these operators using Ili tools to measure all kinds of stuff and we start talking about populations You know the 607 defines it. I mean they got quite a bit of criteria there Like I think you were alluding to at the beginning the data construction The date of manufacturing And it's real tight, like within three years. So, or I guess within two years, like if it's three, it's like it's outside that boundary. Rhett That's for date of manufacture, right? Jason Yeah, and so they're defining population. Like when I look at populations, I'm thinking something that you can treat it, it's all part of the same group. Like they act the same, they have the same attributes. But if you think of manufacturing, like you might have bought something that was manufactured in the same year, but then you installed it three years apart. And they're saying, Hey, this is a different population. So number one, you know, the 607 is putting up a lot of challenges here, because those are really from what I would call a population, those are not really different populations. And what I would care about is the attributes of the pipe are different. The way it contains pressure is different or the way you might assess risk is different. I'd call that a population. But if it's going to be the same across, I would say, Well, you can group those things together. So 607 puts up a lot of challenge in that regard. Now, what the ILI tool does is the ILI tool goes through and measures these attributes like wall thickness. It can sometimes detect seam differences if there's a type of seam, it can detect that. It has some indication of yield strength. And so if you put those things together, you should be able to use the ILI data to say, well, here's a group of pipelines that's part of the same population. I think you can legitimately do that, but there's something lacking, and that's what this paper was all about, was if you go back to the vendors, the vendors for the ILI tools, they don't tell you we're at 95 % chance that this is a population or if this is this kind of difference in the attributes that it's 80 % chance, they're not giving us that spec. It's not yet anyways, it's maybe too early. And so what this paper is about is we looked at a bunch, a big data set, and we tried to ascertain ourselves, what do we think that performances that the ILI is achieving at identifying individual populations. Chris So our, our last chat was around, we brought 1163 in quite a bit and I'll just, I always like to share that a little, right? Jason So there's a good link there too. Chris Yeah, yeah. So, you know, when we look at an ILI system, often if you've looked them for, for any period of time, you're, you should be trained to say, Hey, what's the spec? Right. And when you look at the spec, there is a construct of it that's defined in 1163. And often what we'll find is, is we'll, let's call it sizing accuracy specs. That's kind of one element. But then the other element of it is it's what we talk about POD and POI. Absolutely. And that's normally what we'll see things like, okay, what do we think the system can do? Can it find wall thickness changes? Well, would you normally go to the spec to see if it can. No, it doesn't, it doesn't communicate things that way. Right? It says we can find girth welds, we can find these attributes, characteristics of the pipe. And they'll say we can do this at 90% confidence or between 50 and 90 or less. Jason And metal loss, you know, we can find it. This is how big it has to be. And this is how good we can find it. Chris But now we have to figure out how to interpret that spec to establish populations. Right. And what'd you think? Jason Well, yeah, that's what I was looking at. And you brought up 1163. And kind of the closest link to 1163 is something like POD or POI. Because if you think of a population, it's kind of like a yes or no, like, is it a separate population or not? So it's like a zero or one, it's like a binary. Yeah. So you know what else is like that? POI, probability of identification, like, is it corrosion or is it a crack or is it something else? POD is a little bit like that. Is there a feature there? Yes or no, we're not talking about size yet we're just saying is it did we detect it is it there so this problem is a binary problem but it's got an additional complication it's not just purely binary right tick tock so for example with wall thickness as the difference in wall thickness gets bigger you're going to have a better chance of detecting that and as it gets smaller there's actually a chance where the ili tool kind of maybe misses one of those or maybe it screws that up it doesn't call it correctly so So what you end up having, you can imagine like if you have like a, you know, a graph like an X and Y, like a long one axis, you're saying, well, how big is that difference in wall thickness? And the bigger it gets, the higher that curve climbs and then you'd have a probability. So in some cases, the probability might be 50%. But if the wall thickness difference is big enough, your probability climbs up, could be almost close to 100%, could be 99 % accuracy on that. So that's one complication. The second complication is that we have multiple attributes that we're looking for, right? So we're not only looking at wall thickness, we're saying while there could be a wall thickness change and there might be a seam change. And if you have both of them, then you can be even more sure that these are two different populations. And so your probability of discerning those things can be low in some cases, but it can be also very high if you have a lot of differences. Rhett So what specifically which variables because which ones were you able to look at from 607 and not able to look at from 607? Jason Yeah, we looked at three variables in this paper. We looked I mean diameters an easy one But all the datasets we had were all the same diameter and and that's kind of almost a no -brainer. If it's a different diameter Okay, - Rhett If the ili tools gets the wrong diameter. We have it. We have a serious issue here Jason So that's in there, but we didn't use it in this paper in this paper We looked at wall thickness, seam type or see, they're like seem, yeah, seam type you could say, and then yield strength. We looked at those three. Now, the other ones that are listed in 607, they talk about manufacturing date and they talk about construction date. Rhett You got a tool that can do manufacturing date? Jason I don't. And that's why it's not with an accuracy- Chris -of plus or minus two years. Oh, can't be plus or minus two years, it is plus or minus one year. Yeah. Jason Yeah. Yeah. So it's really, really tight. Now, I think like, I mean, as we implement this as an industry, we're going to have to really scrutinize, you know, why are we using manufacturing dates within two years? And I'm sure there's a lot of debate on that. So I stepped away from that debate. We didn't have that information and we didn't use it in the paper. But I think overall, if we think of like, what are we trying to achieve? What's PHMSA to try to achieve? What is the industry trying to achieve? I don't think like a two year difference in construction date is probably chasing the right thing. Rhett More construction contract than construction data at that point, but that's a different animal. Chris Yeah, and manufacturing. Rhett So wall thickness, long seam, and then you mentioned— Jason Yield strength. Rhett Yield strength. So I wanna ask you two questions on that. With respect to long seam, can you speak to, was it a single oriented MFL technology to you, or did you use multiple orientation? So was it just axially, or was it circumferentially oriented? Jason We had different ILI runs. Some, yeah, some were circumferential and some are axial. Rhett Okay. And so second thought is that not all of the vendors, and I'm sure our audience is aware of this, not all of the vendors determine what you called yield strength the same way that the technology premises different. Um, did you look at this across multiple vendors or was this primarily concentrated in a single vendor? This was concentrated in a single vendor, but you bring up a really good point with yield strength. Maybe I'll kind of iterate a point on that. With the yield strength, the tool is measuring something that has to do with yield strength. And there's some debate on how good it is at coming up with what yield strength is. And what we were doing is we were using that data to determine populations. We were not using that data to say, this is what we think the yield strength is. So if you have the yield strength measurements from a tool, and it has, you know, one group, you know, it's kind of scattered within a range. And then you see, okay, well, now it's moved on to another section of line and now it's scattered, but it's in a kind of a different scatter zone. We weren't saying, well, this is the yield strength that that pipe has. We were saying these two are different enough that we're going to classify those as different populations. So we only use it to that level. And if you read the paper, we have kind of a heuristic, like a mathematical trick that we say, if we do this math, we're pretty sure that these two things are different. And what we did there was we looked at an indication that the distribution, if you look at the yield strength, it's binomial, meaning like it has two peaks or it has multimodal peaks. So that's what we were looking at. And if we think it has more than one peak, or we think it's, you know, not a normal distribution, then we will say it's more likely that this comes from two different populations. Rhett Yeah, that's interesting. What I like about that is that, you know, rather than specifically needing grade, you can particularly rely upon what you called differences in grid, right? And definitely no vendor has a spec on differences in grid that I'm aware of, grade maybe, but not differences in grid. And it kind of opens up the, let's say the possibilities a little bit of what you can look at. Jason Yeah, absolutely. And it's a little bit of a simplification, but for the purposes of trying to figure out well, how many populations do you have? You don't necessarily at that step need to know, well, here's the grade that we're measuring and here's that compares to this grade and our records and so on. I'm not worried about that. I'm just looking for these populations. Can you do it? Rhett So all right, deliver the baby for me. Let's get before I go to the break. Okay. What did you find? Jason We found that if you combine all three of these attributes that in a lot of cases you can be very confident in your distinguishing populations from an ILI took. Rhett So does having multiple ones actually help you? Like I was first I was thinking, oh, one over X times, one over X times, one over X, that would make it more difficult to get to confidence. Jason Yeah, it's one minus one over X. Rhett This is why Rhett Dotson is not a statistician. Thank you, so. Jason Well, you think of it this way. Let's say you had your first attribute and it was a wall thickness change, but let's say it's quite a small one. You know, so you're not really confident. You're maybe 50%, you're like, it's a small wall that we think this might be a new population, but then you dig into the data and then you say, oh, well, there's actually also a difference in the seam type. So that actually reinforces that there's a difference between the two. So it actually compounds in a way that it gets more and more confident as there are more differences. Rhett Yeah. All right. I have lots of questions. I want to ask you when we come back on this audience, we want to dig into maybe where some of the gaps are where this assessment needs to go moving forward. So hang on and we're going to be right back with Jason Scow after the break. Rhett Welcome back to pipeline things right after the break. And so we are here with Jason Skow and we're continuing our discussion on his overly long titled paper, estimating ILI tool performance and identifying unique pipe populations for material verification. Chris A.k.a. Fake it till you make it. Rhett Coming up at the end, y 'all are gonna get the answer to that question but not right now, 'cause Jason, We have special questions for you. Chris We do Rhett The most difficult ones that you Jason Is this the dreaded rapid -fire. Chris They're not dreaded. These are fun. Rhett And I'm not I'm gonna hold them away. Chris This is how people get to know you. Jason I think you're just preparing in case like a fight breaks out here. Rhett It's possible the first one You know- Jason You don't realize that Chris is on my side Rhett Probably so on this one, but let's see the first one. Chris I'm a striker. That's a bad combo Rhett The first one divides entire families for sure. Jason Yeah, shoot. Rhett Apple or Android? Jason Yeah, I mean, I'm not strong when I have an apple, but it's not, it's not, I don't have a strong, I don't have a real strong reason for that. Chris Correct answer, continue. Rhett Doesn't matter, you have an apple, you've already identified yourself. Jason I also have a Mac computer as well, so it's okay. Rhett Oh, you're way deep, you're all the way. Jason And you know what, you know what, other people in the company- Rhett You should have just straight answered Apple. Jason Other people in my company, they have the Microsoft laptop. And then, you know, when there's an update, like every 10th time, you know, they're like blue screen, I'm working, okay? I'm working. Rhet Back to rapid fire. Back to rapid fire. Football or futball? Jason Football. Rhett Okay. Good. Curling or hockey? Jason Hockey. Rhett Did you play? Jason I did not play. I'm one of the few Canadians that didn't play hockey. Rhett I started to put in there whether or not it was Oilers or Flames, but I figured that's probably – Jason It's Oilers, but I'm a fair weather fan. I'm watching the you know, I'll watch the Super Bowl I think I know three of the four teams still in it. So I'm a fair weather fan Rhett Yeah, do you think do you think that hockey is rigged like people think the NFL is rigged? Jason I don't think hockey is rigged. No, it's there's too much variability, you know, If you think of a, okay, this is a little off topic, but you think of a like a chess board. Rhett He's about to give us the district answer. Jason A chess board, you know, it's, you know, the number of squares, 64 squares in chess board and how many different possible moves you can make. And it's like more than the atoms in the universe type of thing. Just imagine you didn't have squares. Okay, that's like what a sports play, sports game is like. There's an unlimited number of possibilities to suit. That’s why it's fun to watch. Rhett Historically, sports games have been fixed throughout history. Nevermind, all right. Skiing or snowboard Jason Skiing. I'm not coordinated enough to go. Rhett And since I heard that you're into like martial art stuff Is your first incline inclination to punch or kick? Jason Well, I've been training and jujitsu recently, so I'm learning to not punch But my inclination is still a punch like if you were to come at me right now, I'm throwing man Rhett Live, you saw it here on pipeline things Rhett came at Jason Jason But you know I started jiu -jitsu like a few months back with my kids and I realized you know I'm just getting out of shape. I'm not you know I'm not gonna win any competitions, but I get I get tapped out 20 times a night It's embarrassing, but I'm getting in shape and my kids are learning Rhett I started working out too and I did something called body pump and The first one I didn't walk for like four days after and they were 60 year old women just destroying me in class So – Jason I feel that pain. I feel that pain. That was when I first started, I couldn't do four minutes, but now I can do 20. I'm getting up there. Rhett Alright, catching up the audience going back. So when we finished up our last segment back to the show, we were talking about your conclusions from your study and the conclusion you reached was that with the variables for looking at populations, namely wall thickness, differences in grade and seem type. The combination of those means that we can have a very high confidence in the identification of a population based on our ILI data. Jason Not so fast, my friend. Not so fast. Rhett Oh, he's coming at me. And if he's the one to punch, I'm ducking. No, no, no. Jason Well, you're right about that. You can be very confident. But you know what, we think of these binomial problems where there's just like two outcomes. We actually have four different possible comes from that because you have a prediction from the tool, which is like, it's a new population or not. And then you have what's reality, it's a new population or not. And so you can call it to be a population and be right. Or you can call it not to be a population and it is another population. So you have kind of like a four, a two by two matrix, like four different kind of possible answers you can have there. Yeah, exactly. You're drawing it right there. Rhett Sorry. Jason Yeah, and when we have an ILI tool that finds a population, we're very, very good at, you know, determining what's that probability that it actually is a population. And that's one kind of performance. But the other one that you have to think about is, what if the tool didn't call any different population? What's the chance that there's another population buried in there? That's a different kind of performance. So in statistics, one is called, you know, recall, and one is called precision but it's just kind of measuring those two different errors. Chris So what's the what's the standard right? So if we look at 607, one of the first numbers we see in there outside of like one per mile or up to 150 digs, we see another number and that number is 95%. And their descriptor is confidence. Yeah. So what do you have to say about that? Are you are you proposing that in this paper, a means to establish a 95 % confidence in your population? Jason Yeah. Well, that's an excellent question, Chris. And I think I know where you're going with that, which is the statistics in the way it's written up in 607 are a bit loose. They're not quite giving us the performance you know, as an engineer or a statistician to come up with, you know, how many digs do we need to do? Like, how do we get to that performance? And I'll tell you why. You could be 95 % confident of something, but your performance could be terrible. Like, let me give you an example. I'm never going to achieve this performance, and I'm 95 % confident that I'll never achieve it. So what am I really saying there? Chris You fake it till you make it. Jason I'm faking it till I make it, exactly. Rhett Wait, say that – Jason So you actually need two numbers to define a performance. You need to say, I'm going to achieve this level of performance, like maybe 80%, you know, like the ILI tools, 80 % of the time, this is what I'm going to get. But then if I repeat that over and over again, 95 % of those times, I'm achieving that. So you need two numbers. You need a performance, like a probability, and then you need a confidence. And with the 607, they just say 95%, they don't give you two numbers. And so we have to interpret that. So here's how I interpret it. I think it's the average probability of defining a population. I think that's what they meant when they said 95 % confidence, even though it's a bit ambiguous from, I think, a straight statistics perspective. Chris Well, we appreciate that because there is a lot of room for interpretation. Rhett I'll be honest, I'm still thinking about what that means. Jason Well ask me a question, Rhett? Ask Ask me a question about it Rhett I want you to say it again, what does it mean that it's the average probability? Are you basically stating that 95 % of the time when they call a population, it will be correct? Is that way? So if I were to look at 100 calls, I should expect approximately 5 % of them to not be in layman's terms. Jason In layman's terms, yeah. And for each one of those performances usually have an upper and lower bound, like a confidence Right around an average performance and so I'm interpreting the 607 to just be the average performance with no bounds on it And I'm going to as an engineer working, you know for an operating company I'm going to impose some bounds and then I'm gonna try to rationalize why I picked those bounds because the 607 doesn't give It to us. Chris We have we have a saying for that. Yeah, define and defend. When there's room for interpretation – Jason I think that's what you have to do, right? I think that's that's your best option, your best strategy. Chris You write a policy and then you defend it. Rhett And also sound a better than punch and block. Chris So I mean, to put some— Jason I define it with these five fingers. Chris But again, we threw a lot of terms up here, Jason, right? And, you know, for somebody who might not be as versed as yourself or other people who are doing statistics around assessments, I feel like it would be helpful for us to kind of bring some of that back down. You know, one of the things that we'll see in, in ILI performance specs is they'll say, they won't use probability. They'll say certainty. Yes. Yes. And so for example, a lot of times the way we normally try to interpret that is it's, how many times was it correct? Right. - And so you would say this has an 80 % certainty or 90 % certainty, or that's where a lot of times we'll see somebody say, well, eight out of 10 or nine out of 10. - But And we also see that term of confidence, right? And so normally, I think we talked about on the last one, right? Where it's where, you know, normally we try to communicate things as intervals and you now probably said, maybe it's the average of that. And a lot of different people might say, well, I want to use confidence intervals. And if the upper bound says that it is possible to achieve a confidence level of 90, then I'm happy, then I'm good. Jason And bringing it back to the paper, we talked about three different measures that we're defining performance around. So in the paper, we define how to come up with a confidence around each estimate. So for wall thickness, you have, you know, an average performance that you have an upper and lower bound. Chris Perfect. Jason And then for a seam, you have, you know, you're, you're predicting the seam, you know, if the seam type change, and you have a, you know, a probability up and down or confidence around that, because It's just a binomial confidence like Clopper Pearson, like rate at 1163. And then the same thing with yield strength. We have a confidence around making the call. And in that one, like I was explaining earlier is a heuristic rule. So the heuristic rule is not always right. So it's kind of, you know, has a confidence. Sometimes it's right, sometimes it's not. And we could come up, you know, this is a first paper on this topic. We could come up with a better rule. We, you know, you could look at, you know, machine learning and you could look at grouping populations by several metrics. We used a simple one here, and we could probably improve on that, but this is the one that we published, and we think it's straightforward and easy to interpret. Chris And again, one of the reasons why I really appreciate what you guys did here is because often when an operator or us as consultants, when we're asked, "Hey, can you help with this?" We need things to grab onto to help us define so that then we can defend, right? And this is one of the works that people can turn and start leaning into. And one of the other things I'll put out there, right, is it's, when an operator is reading or a consultant's reading 607, again, I think paragraph E, and they say, you need to define populations with all these criteria, you would say, well, where does it say I can use ILI? Can I? Jason Yeah, in fact, in the QA, there's actually a section where it does say you can use ILI, right? Chris That’s one for RIN one of FAQs and I think it's FAQ 21 Rhett He's so bad about it. Miss producer. Chris It's FAQ 21 of batch one of RIN one FAQs and in fact They do say that right they say inline inspection can be used to establish populations And so this allows an operator to say okay great. How can I start thinking about? Using ILI for populations and you have the framework from a regulatory perspective to do it. Jason Yeah. And you know, I called them QAs, but yeah, they're FAQs, right? It's a better way. So you know what else they have in the FAQs? What's that? What's that? They have this really interesting one that says, what if I don't know the data construction? What do I do then? And you know what the answer is? You just bucket them into like an unknown. Yeah. And you just go from there. And you know, unfortunately, this has- Chris And if you find two that are different, those become new buckets, it's now you have to go into an opulent sampling, alternate sampling. Jason Exactly. But you get into this difficulty of if you have more data, you're going to split them up into smaller and smaller groups because of the requirements of construction and manufacturing dates. So you're going to have lots of populations. But if you don't have the records, you're going to start grouping them into bigger populations. And the thing I don't like about that concept is it gives you an advantage to not know. And I think generally when you have regulations and best practices, you want to encourage finding out more records or digging harder and that you get some benefit from it. In this case, there's a there's a decent chance that you find out more and it's actually more costly or it's more -- Chris Or it's it's not more expensive, right? So think about the dynamic segmentation along the right -of -way, unknown, known, unknown, known, unknown, and if you don't know what the unknowns are and you put them in a bucket and it's less than one mile, how many digs do you do? Jason You know, I love you thinking 3D, Chris, because you're right, it goes both ways. Chris How many digs do you do? If it's less than one mile, you can have five different unknowns and it's one dig. So you might've only identified one of the four, one of the five unknown calculations. Jason Absolutely right, yeah, there's uncertainty I was just saying one side of it and Chris, you were thinking multi -dimensional. Rhett And I was on the third side, I was like, it's not just limited to this. What if I do a hydro test, then run a crack inspection? Do I have to repair all the immediate crack inspections, even if I just did a hydro test? Don't answer that question. Jason My paper doesn't cover that. [Laughter] Rhett I know, but that's a penalized potentially for getting more information options. So, I thought that's where Chris was gonna go and he didn't. So, Jason, before we go, last question. Do y'all answered the fake it till it make it? That came out. I heard that. Chris It 100 % does. Rhett That's what I thought. That was around the confidence. Jason It was 95 % confidence. Chris The whole point of it is that you could say it's 95 % confident, but you don't establish what the performance is. You're faking it till you make it. Rhett Got it. Where do you want to go next in this? Or is this topic pretty much covered for you? Where do you see the future of 182607 in this evaluation with ILI? Chris In ILI related work. Jason Yeah, I mean, what I think is we're probably gonna come to more formalized performance specifications around defining populations. I also think we're gonna be able to define a lot of populations using ILI tools. I think that's really gonna help us out in this effort. So we have, you know, I think the follow on for me personally is, you know, I'll look at some of these performance specs that we defined. I think we can refine them. You know, one thing that you always gotta think about when you're coming up with specs like this is like, you know, what data did you use? Did you collect enough data? If new data emerges, how does that change what you've done? So we haven't done the complete study on this. We were working with a great operator. They had a very good data set, but does it cover the full spectrum for the industry? The answer is no. This is more tuned to the operator that we were working with in their data set. So you have to keep working at this and refine it. So I'm gonna be on side with that. And I'm also on side with helping the industry come up with more generalized performance that would apply across, let's say, all the operators in the industry. And I think, Rhett, you were mentioning last time we talked on this, that things like the diameter matter. 'Cause different tools at different diameters, they have slightly different performance around these attributes that we care about. So yeah, we need to include more diameters. We need to include more different types of runs. We need to include more different types of vendors. But the concept is ILI can really help us to find populations and it's a great tool and we should use it. Rhett Fantastic. Jason, I want to thank you for being on and we have one last opportunity for you which would be unique to our PPIM guests as a part of the series. Okay. What question would you like to ask Christopher and I? Jason Oh, live? Turn off the camera, I got 10 questions. Rhett No, you only get one. And she's gonna edit out the silence while you think. Chris And you better not ask the question that you asked and we were all offline. Jason I'm not gonna ask that one. Chris Yeah, that could fracture our relationship Jason I would say uh, you know last time I was on you know Rhett you were It's just to me. You yeah, you were you were saying, you know that uh, Chris and I have a bromance and you felt left out That's why I sat in between the two of you. Chris Is that why you sat between us, Rhett? Rhett And I sat in between you. Chris Is that why you sat between us? Because normally our guests sit between us and I'm on that side. Rhett Jason, the answer to that question is, having met you in person now and sat beside you for the duration of this episode, I understand the bromance, I get it. Jason And I was gonna say, you know, welcome to the club, it's a Manaja Troi now. Rhett Oh my gosh, you heard it first on Pipeline Things. Jason, thank you for joining us. That is the conclusion of our PPIM episode on Jason Scow with the overly long title, "Estimating Highlight Tool Performance on Identifying Unified Populations for Material Verification." Thanks and see you again in two weeks. Special thanks to executive producer, Sarah Etier , Jason Scow of Integra Engineering and the Work Lodge for hosting us while we film. (upbeat music)