Rhett Third-Wheels Christopher’s Discussion about API 1163 Level 3 Validations with Jason Skow

Rhett Dotson So curious question. What audience does a study of the influence of adjusting people's income have to do with pipeline integrity? If you've never, ever been curious about that, you should listen to today's episode where we talk about the influence of truncated data, what that means on a level three validation and API 1163 and if you're curious, well Rhett, I don't do an 1163 level three validation. On this episode. We talk about maybe why you should and can you say, Christopher, hello maop reconfirmation. Join us. All right welcome to today's episode of Pipeline Things. I am your host, Rhett Dotson. My co-host, Christopher De Leon. And we are excited to be with you as we continue our series on IPC publications from 2022. And I like what we have coming today. It's, it's one that, that, makes me it makes me think, which I like, Christopher. Christopher de Leon And it’s with someone we like. I like him. Rhett Dotson I think this person likes you, too. I mean, they know they told me the whole reason. Christopher de Leon Are you jealous? Rhett Dotson Maybe a little. I mean, when a guest says that they're filming because of you and I don't get mentioned, then it's. Christopher de Leon Look, I’ve always told you brother, you are the spotlight. Like people always wonder, like it's D2. So who's D1 and who's D2? Rhett Dotson Look let's not reveal that to the audience. Let's leave it for the time being. When we were preparing for this show, I had a thought. Do you read? Christopher de Leon Do I read? Rhett Dotson Do you read that? Not can you read? Do you read? There's a difference. Do you read? Christopher de Leon He's spicy today, I wonder what he had for lunch. Yes, I read, but it depends on what? I read for fun. Rhett Dotson Cool. I read for fun too, but my, my, I've been on behavioral economic theory for a long time. Christopher de Leon Yeah, you have mentioned this for a while. Rhett Dotson I know, I know. Christopher de Leon You want to understand people, people are complex. Rhett Dotson I do. So, one of the things that I read recently was the hidden cost of free. Right? So, people don't often think about the hidden cost of free. But from a marketing perspective, as you drop the price down for something, there's not a linear response. So, when you get the free, people just behave completely irrationally because like they don't justify the cost of free. So, in your own life, think about this. Christopher de Leon I've got a perfect example. Rhett Dotson Do you really? Christopher de Leon I mean, it’s not free, but it's low cost, right? It's just I don't know if you. I've heard like this family got allergic reaction when they're at Disney and, the lady they're passing because allergic reaction. And they tried to sue and apparently, they lost because when they signed up for the Disney Plus app, it says you cannot sue us for anything, basically. And so, they lost a totally disconnected thing. I signed this, yes, I agree to your terms for the app, but when I go to Disney, you kill me because you didn't pay attention to my allergies. You can’t sue me now. Rhett Dotson Wow. Okay, that was a little different. I was thinking, take it back two notches. Like my wife likes to go to Starbucks. And she went and waited an hour in line for, like, the Starbucks free, the free holiday Cup. To think I was like, you waited an hour in line for that, that thing probably cost five bucks. Like, I can't give up an hour of my time. She's like, well, I was already in line, I was stuck, it was free. I went with it. You know, people behave irrationally when you set the price of something to free. Christopher de Leon This timing is going to be perfect. Okay? PSL is real, pumpkin spice latte, bro. It is real, right? It's kind of like when Chick-Fil-A brings back the peach milkshake. Like it is real. Rhett Dotson So, I think I think it's going to make sense when we bring on the guests and we get into it. But as he was talking. Christopher de Leon The cost of free. Rhett Dotson It's not the cost of free because nothing's free in our industry. Christopher de Leon Opportunity cost. Rhett Dotson But yes, there's an opportunity cost we often don't consider with things. Free is a good example in our own lives and you drop it for free again. People will often get paid more for something because it's free, and marketing takes that into account. Christopher de Leon And everything that we do, obviously. I mean, it's probably a little bit of a spoiler, right? Just to connect the dots like it might not be free, the lack of effort, it will be something you don't have to lean into it, or you don't want to go through the effort. Rhett Dotson So, I guess it's time to bring in our guest and we can let them know. Do you want to introduce them. Go ahead. Christopher, I want you to it since obviously you’re his first love out of the D2 group. You should bring on the guest this time. Christopher de Leon All right. Well, hey, guys, I, I get I have the privilege of bringing on Jason Skow. He's, I consider him, a good friend of mine. Brilliant. Really plugged into Ili. Really plugged into the numerical methods of learning how to use Ili. So really looking forward to having you on the show, brother. Welcome. Jason Skow Hey, thanks a lot, guys. And, just to be clear, Rhett, you know. Rhett Dotson Jason, it's okay. Don't defend it. It's okay. Jason Skow I love I love Chris, but it's like, because I know him better and, you know, from what I can tell, like, lots of potential with you too. Rhett Dotson You know what? I will actually relinquish my role as the host on today's show, I will just be a guest on the show, as I watch Jason and Chris have a friendly conversation. Christopher de Leon Oh no, I hey yeah. Again, we always we haven't had a, a segue where you and I have like, talked about ongoing. So, a little deep dive into our business. Right. So, I may know people first, but once they meet this guy, they stop calling me. They always call this guy. So, I totally expect that to happen. Rhett Dotson I don't I don't see that. I don't see it because now I feel I'm so hurt by Jason's love for you that I don't know if we can get past that, that gulf. Yeah, to build that bridge. But, Jason, you're our guest today. Thank you for joining us so much. We brought you on because of your IPC 2022 paper, Estimating Measurement Performance with Truncated Data Sets, which is a mouthful. But I don't think it's complicated. So, our audience I want you guys to hang on because what Jason is going to talk about today has a lot to do with topics of interest for you, namely how do you validate. And ILI 1163 and what those methods mean. And it's going to be a jam-packed episode, but it's also going to be a lot of fun before we get too deep into that, Jason, tell us a little bit about yourself. Please introduce yourself to the audience if you don't mind. Jason Skow Sure. Yeah. No, I'm happy too. And so, my name is Jason Skow. I've been in the pipeline industry. Well, I say I should say oil and gas industry about 23, 24 years. My first role was as a plant engineer in a refinery. But then soon after that, I moved into the pipeline world, and I worked for a pipeline operator in Saskatchewan. I was doing I started off doing distribution, designing town border stations, regulator like down stations and, you know, new loops inside cities and towns, but then moved on to integrity. That was when they were expanding their integrity group. I think they had 1 or 2 people in integrity and they needed like a third, so this was a much smaller integrity group than you find now, even at the same company. And I think the same has been replicated in other operating companies. So, I was doing pipeline integrity, I was doing pigs and digs. I was out in the field every summer. I spent probably about two months’ worth of time out in the field, you know, doing fitness for service evaluations, doing modifications, sometimes some cut outs, and just making sure the integrity of the pipe is what we expected. So that was my life for a while, I loved it, I loved being out in the field. I love working with, you know, instrument techs. I love working with the welders, the operators, and I learned a lot. You know, one thing I learned was that although we're working with a simple asset, you know, it's a pipeline. It's like a tube kind of, every time I went to the field, something different. Every time something is different. There's so much variety. Like, reality is so complex compared to our simple models. And so, I really gained an appreciation for that. And I really gained an appreciation for just the team of people that is necessary to make everything happen. And I was just sort of one part of that team, like one step on that sort of ladder. But everyone together sort of made it happen. And I and I tried hard to, work with that team, gain their confidence and really understand what they were doing in their positions and stuff like that. So, it was fantastic. After my role there, I moved on to a more of a research role. That was see, for technologies. I was working basically for the Alberta government at that time, did that role for maybe eight years, and then found that, you know, sometimes government can kind of handcuff you a little bit like, you know, government is not the right place for innovation. It's not the right place for business. It has a role. And I, you know, played that role for a while. But then I decided, you know, we've got a confluence of market forces that are demanding an answer. I don't know if you guys felt the same, but I would summarize it as simply, democratization of the CPU. That's cloud computing. Meaning somebody like me and a small group of, you know, professionals could have access to computing power as big as supercomputers out there, you know, and for a very low cost. So, democratization of the CPU, we have machine learning type algorithms that are becoming mainstream, meaning engineers can use them for practical purposes. They're not research projects in a, you know, scientific lab or, you know, university lab somewhere, you know, that you can just use you can download a library, a Python library or something like that, or maybe even a, you know, Excel plug in stuff like that and becomes usable for practical purposes. And then thirdly, at the same time, aging infrastructure is getting older. We can't replace it in your fast enough that we, you know, that we need it's a critical part of the industry, of the economy. Even. You know, I've heard some economists even claim, you know, energy is the economy. So, you know, we have such a reliance on, strong, secure source of energy just to make everything else work. You know, like you don't have an iPhone without good energy. So, you know, just a core element. And the infrastructure is aging. So, what could I do to contribute to that world is, you know, hopefully use computing power, machine learning algorithms and experience and integrity services to keep the asset in shape, running longer, past design life, keep it safe. You know, protect people, protect the environment. And that's what we're trying to do. Rhett Dotson And so, in all of that, one of the things I'm most curious about is do you remember when you first met Chris? Jason Skow I mean, it was a PRCI. Christopher de Leon It's always memorable. You know this, Rhett. It's always memorable. Jason Skow And I think I think like the very first time I met him, I was sitting. So, I was in a PRCI. There was one of the projects was being discussed. Rhett Dotson That's enough that I don't need. Jason Skow And then. Well, I got to tell you, I got to tell you that I left. I had to go take a call, I came back, Chris is in my seat. I think that's the first time I saw him. But then after that, you know, we started chatting and, I think at the time, Chris, you were with Rosen. You know, so, yeah, we had a lot of good discussions. And, you know, I think our or, you know, our friendship kind of grew from there. Rhett Dotson Yeah, yeah. Well, thank you. Jason. So, for indulging that's going to happen a lot throughout the podcast. Christopher de Leon And I know I'm going to do that. I'm going to do that the next place you go, wherever he goes, he sits down. I'm just going to sit in a seat for a minute. And I'm going to write Jason, on it. Jason Skow I like I mean, it's a free seat, right? There's a free seat like. Rhett Dotson That's true. There was a cost of joining it. You just didn't know what I was going to give you. So, well we had a guest that joined us in the podcast, too. The cat totally jumped up in the background. That was, that was great. So, we have, we have, we have a new guest. So, Jason, tell us a bit about this paper. Layman's terms. What drove you to do this paper, like, set the stage for me a little bit? Jason Skow Yeah, sure. Well, this paper is playing off, one of the techniques that's listed in the API 1163. And it's about validation of measurement tools. Rhett Dotson And, you know, when you say validation rules, 1163 is all geared towards ILI tools, right? So, when you say measurement tools, you want validation of ILI measurement tools. Is that how the audience understands. Jason Skow Yeah. Yeah actually, yeah. 1163 is all about ILI. But in fact, the reason I said measurement tools, it's very astute of you to pick that out, because I've used the same techniques for other types of measurement tools, really, really what it is. It's about anytime you have a measurement that you do a lot of and it's cheaper, and then you have a reference measurement that you do, fewer of that are more expensive. Anytime you have that situation, it could be in medicine, it could be in refineries, it could be, you know, radiography. It could be a new technique that we borrowed from NASA to try to measure stuff on our pipelines. Whatever it is, if you have those, those, conditions where you have one measurement, where you have lots of it, and another reference measurement, better quality, and you have fewer of them, they're usually more expensive than you can apply this, technique for validation. However, in our industry, you're right. This is applied primarily to ILI tools. But you could imagine you know, if we come up with a new technology or we're testing a new technology. And the question was, is this ready for prime time? How good is this compared to other technologies? You would use the same techniques. So, it is generalizable to more other types of measurements as well. Rhett Dotson All right. So going back, I'm sorry I interrupted you. You were setting the stage, and you said so we have an example, an 1163 follow up. Go ahead. Jason Skow Yeah. Right. And so, what we're what we want to do is we want to validate measurements. And 1163 lays out three methodologies to validate. So, they're called level one level two and level three. And the one that the papers are about is level three. You know, I was able to contribute to the appendix of API 1163, the current version and one of the examples using level three. So, I kind of had that under my belt. I had been thinking about those level three, measurements. And then this paper kind of came from that because I was working with an operator who is really interested in applying level three. And so, we tried it with the data set. We encountered some difficulties, and truncation is one that I'll get into. But basically, the idea of the three levels of validation is and I think Chris was alluding to this at the beginning, the intro to the podcast is, you do the amount of work that's, that is, proportional to the cost of making an error. Right? So, you know, if you need a very quick validation and maybe the error, you know, if you get it wrong, it's maybe not that big of a deal, or maybe that's something that you can overcome. Or maybe you have some safeguards for that. Well, then you just kind of stop your effort if you don't continue. And do, you know, a PhD research project every time you just do, it's necessary. But in some cases, you may need to be more accurate. You may need to have more confidence in the numbers that you're presenting, maybe presenting those to upper management, to regulators or something. And so different levels of validation are appropriate under different circumstances. And this one was level three. That's the more detailed one. And then the paper focuses on the problem of truncated data, meaning if some of the data is missing out of the data set, can you still do it? The answer is yes. And the paper kind of outlines how to do that. Christopher de Leon So, hold on for clarity just for the audience. Just curious. So, when you say you do you worked on the latest version. We're talking about the 2001 version of 1163. Jason Skow 2021, I believe. Christopher de Leon 2021. Yeah. I'm sorry. That's what I meant. 2021. Yeah. Rhett Dotson You just went back two decades. Christopher de Leon Yeah. I'm sorry. Yeah. 2021. Sorry about that. Okay, just to be clear. Rhett Dotson So is it fair to say, Chris, and I'm interested in your opinions on level one, level two and level three, I feel like I see very few level two assessments. I think I can count on one hand how many level three assessments I've seen operators do. What do you think about that? Christopher de Leon So, I'll add as part of answering your question I'll add Jason. Right. So, 1163 I think is fundamental for pipeline integrity. Right. It outlines the process in which operators should use in line inspection. I won't get into all that, but it's a lot more than just a validation component. Right? And in it, one of the things that I always talk to our clients about is that the risk that the pipeline poses and the cost of getting something wrong as it ties to that risk should be a factor in which you choose. Am I doing level one, level two and level three. And as part of that decision process, not only is it the risk of getting something wrong, or the risk that that line carries the consequent side of risk, but it's also how much of an understanding you have with that ILI system, or how much information you can get from it. So, for example, if you say, hey, we've used this MFL ILI system for the last four years, you know, we've done, you know, 140 digs on it. And I've as it says in 1163, you have maintained a database of my findings in the performance. Then that's where level one can step in and you can say, hey, I've already done a ton of work. I'm going to pull that work forward because all the essential variables as defined in the ILI system remain. And so therefore I'm going to use all that experience and say I can do a level one on this pipeline commensurate with risk commensurate data integration commits are my experience for that ILI system. Rhett Dotson Right okay. Christopher de Leon Where level two comes in is where you feel like you need to supplement a bit, right? So maybe you don't have as much experience, or the risk is a little bit higher. You say I'm going to go do a handful of digs, right. And then there's a statistical means I won't get into all that. But there's that, that what I'll call a straightforward statistical approach to where you can say, if I just do some work, it gives me an increased level of confidence that I can use this data, and then I'll pause there. Rhett Dotson Okay. So then obviously level three is one step beyond that. Christopher de Leon And the level three being that you are establishing the ILI system's performance specifically. And so you have that and you're establishing it for that ILI system. And then there's some uncertainty like you kind of get to I'll leave it at that, I'll leave it that vague and say each operator can use what they need. Rhett Dotson I'm curious what's driving your operators to level three? You mind shedding some light on that? I mean. Jason Skow Yeah, yeah. So, you say you can count on one hand the number of level threes. For me, it's a, it's a bit more. And possibly because I was, you know, helping with that example that's in the 1163. So, so people kind of touch base with you maybe more frequently. However, I would agree with you that it's not the most common assessment. The most common assessment I think is level two and level one. I mean, level one I think is sort of default level two done all the time. And then, level three is less common, I would say becoming more common. And so, to be more direct in your question, okay. Why level three? I think there's several reasons for level three where you'd want to go that way. One of them is, for example, let's say you don't have that experience that Chris was talking about with, with a technology stack. You're starting something new like this could be, for example, would IMAT come out and it's and it was new at that time. There would be an ILI vendor spec. But you might also want to say, look, we're going to we're going to do a little extra effort here because, you know, we haven't used this before. We're staking a lot of our reputation, and you know, projections with social license, I mean, social license and so on. Yeah. So, there's a lot riding on this. And now other technologies, we have those, you know, we have lots of experience in that. But this one's newer. So, we want to do a little bit of extra work on this to find out. You know, how do we think it's performing independent of what the vendor is saying? Or, you know, you can combine the two in a level three, you can combine the two, you can use the vendor. Christopher de Leon So, hold up before he says anything. So, we recently talked to Bruce or posted, and we were like you know EMAT is like the baby. You know, it's like it's not the first child that you hold it to all these high standards. Right? And then you run it and you're like, well, I've kind of been through this, you know, we can manage this a little bit differently. I have a feeling that's where he was going with it. Rhett Dotson No, it wasn't. EMAT would have been a funny target, but no, what I was really thinking is y'all are mentioning the risk, right? I got it with a new, I think, new ILI system. Easy. I think for a moment to think about the risk proposition now actually where I went, and I don't know if you're familiar with updated regulation in the United States or how familiar you are with it. Jason. But I was thinking about the okay, so I was thinking about MAOP reconfirmation via ECA e and should something like a level three because of the risk associated with that be something operators should be thinking about? Christopher de Leon Why are you pushing my opinion? Rhett Dotson But because it could occur to me as you talk about this, right. So, I mean, one of the riskiest places we might have would arguably be MAOP reconfirmation via ECA and should you do a level 1 on that Christopher de Leon So, I'm trying to be vanilla on these podcasts and get our speakers to not be vanilla. But I'll say this right. So, in the US, unfortunately I feel like how to use level one, level two and level three isn't always well understood. Which is why I'm. I was happy that you guys won the project at PRCI to have the guidance document for using level one, level twos and level threes in a practical manner for inline inspection, but it kind of goes beyond that as well. Right? It's like, I feel like level ones are used inappropriately because the appropriate level of data integration isn't there. Right. And I don't think it's fair for industry. I won't just point at operators because 1163 even says you must name the responsibility between the operator in the aisle and the service provider as to who's responsible for what. And too often that's a big gap, right? So, 1163 outlines that the, the, the essential variables, the process by which it a performance spec was established needs to be documented and available for review. How many times of operator have operators reviewed that and understood how that spec was established to give them the confidence that they could say, you know what, I'm comfortable doing a level one here, right? And so, when you bring up that topic of MAOP PCA. Yeah, it absolutely says one thing, why are you using this tool? How is it performance based? What work have you done on it? What's your data like? This is a whole podcast on its own. I know I do that, maybe. Why would you do that? Rhett Dotson We’ll bring Jason back to talk about MAOP Reconfirmation and the use of level threes. In that case it just you sparked it. I was looking for a case, a case a reason and that's it. Jason Skow So that's an excellent example or an excellent example. And you know just to kind of maybe add to that example, let's say you were looking for the, you know, confidence that you could go with the vendor spec. Well, how about you do a level three? You might calculate something that's the same as the vendor spec. Then, you know you can replicate the vendor spec. Then you have higher confidence. You can rely on your own data set. I've even seen cases where the vendor spec is wider than what you calculate with level three. I mean, it doesn't often happen, but it can happen. It does happen. Some tools perform very well. And then if you're using a narrower spec, you can that's, you know, a more efficient program because you have tighter tolerances. Like another one I would mention, because this is becoming more and more common is, many operators are doing quantitative risk assessments. You know, the one with Monte Carlo where they're trying to figure out probabilities in that case. In that case, the measurement performance is a key input. And so, you might want to do if you're going to do that kind, that style of a risk assessment, you may want to do a level three, performance evaluation. So, and that becomes an input to your risk assessment. So, I've seen the I've seen that application many times where rather than just rely on the vendor spec or a level two, which is more, you know, can I reject the vendor spec or not? You calculate it with a level three that becomes an input to your next step in your process. Rhett Dotson It makes a lot of sense. So, I'm seeing the cases behind level three. Before we break, you know, our friend Bruce said something and I'm just going to ask you, when you do this, you should statistically show that all the tools have the exact same statistical performance, right? And on that note audience. We're going to take a break. When we come back, I promise. We are going to dive into what these gaps and truncated data are. And you really want to hear the story behind how Jason got there and the application to level three. We will be right back after a quick break. Welcome back to Pipeline Things, where we continue our conversation with Jason Skow on the truncated data sets and estimating measurement performance. So, and that last the beginning part of the episode I think we did a good job really establishing level one. Level two, level three and what drives a level three. I appreciate you doing that. Now, Jason, I'm going to ask if I need you to deliver the baby for me. Right. And the Reader's Digest in this publication cut straight to the chase. What did you find and why does it matter? Jason Skow Okay, well, I mean, can I add one thing to level one, level two, level three? Rhett Dotson Well, I know Chris is going to let you because you're his favorite. So, I'm not really going to get much of a say. So, you might as well go ahead. Jason Skow Okay. Well, it kind of leads into what you're asking me. So maybe that helps. For context. Level one relies on experience. It doesn't rely on new dig results from this ILI run that you've currently that you're currently evaluating. So, the data requirements are very small. The level two you start to do some digs okay. Pigs and digs. That's what I used to do I was out in the field doing that. I was the engineer on site making calls, you know cut that pipe out. No, you know, that's clock spring. Whatever. So, level two is you're going out to do digs. You're measuring this meticulously as you can out in the field. You're trying to find out, okay. The ILI called this, but then what did I find when I was there? You know, in person. What did I find? And the test that you do on level two is not saying. I'm for sure confident that the spec was met. What you're saying is, I'm not confident that I can disprove the spec. So therefore, I just go with the spec. So, if I collect data and the data kind of disagrees with the spec, but I don't have very much data, I'm still not confident that the spec was wrong. That means I still go with the spec. I just default back to the spec, unless. Rhett Dotson It's almost like innocent until proven guilty. The spec is innocent until proven guilty. Jason Skow Exactly. And that's the default. I mean, the vendors know their tools. This is the spec they provide. So, then you kind of have to say, well, if I'm going to disagree with the spec, I better have lots of evidence to show that. However, the common case is, is that we collect data. It's very expensive to collect, so we don’t collect that much. It's quite hard to get to be confident that the spec is wrong. So often we just end up defaulting back to the spec. Now, does that mean that the spec was achieved in your ILI run? Absolutely not. It just means that you don't have enough data to reject the spec to reject it. Christopher de Leon Yeah, exactly. Yeah. That's no reason why we say it right. It's you don't have enough evidence to reject it. And so ultimately right. It puts you in a tough spot. Right. It's like, hey I have some data that's good. Some that's kind of not sure. But at the end of the day, I don't have enough to reject it. If you want more to reject it, bump it up. Jason Skow Yeah. And so, then level three is, where you take the data that you have, and you try to derive what the performance was from that data set. Rhett Dotson The actual performance. Jason Skow The actual performance in your run. Because, you know, a toolkit can be different in one run versus another. That can be circumstances that, that, that lead to that. Some of it would be unknown. You know, why is there randomness? But essentially, it's not 100% constant across every run. I mean, that would be great if it was. How did it do in your run? How did it do on this pipeline? We're assessing this pipeline’s integrity. And so that's really the answer we want to know. And level three is very specific to the pipeline that you're assessing. Okay. So now why does it matter. When I was doing this assessment, it was with an operator that was testing different ILI tools, and they were comparing the performance of the two tools, deciding on which one they were going to run on other pipelines. That were similar, similar type pipelines. And so, they wanted to know the performance, not just what the vendor told them, because, you know, you just go through the catalog and say, well, this one claim slightly better, this slightly better pod, slightly better, you know, tolerance bounds for this category of whatever. You could just pick it from that. But what they wanted to do is they wanted to look at, you know, what can we calculate on the runs that we're doing? We wanted the evidence ourselves. We wanted to come up with the performance results. So, we're doing level three. And we encountered a difficulty, which is that data can be truncated. And what truncated data is, is when you have a data set, you can imagine a unity, plot where there's like dots, kind of like along a 45-degree line. You often see that, but just take the lower part of that. Just cut it out. Just disappear it with the reporting threshold. Christopher de Leon Which could happen for a lot of reasons. Right. Like to make this practical like you want you want to have, the best use of your resources, right. So, it's like you're not you likely won't go dig things that are benign or of no interest, because it's not the best way to use your money. So, you probably went and got the nasty stuff, which means you're probably on the upper end, right? The upper left quadrant of your unity plot. So, what do you do with the lower stuff? Jason Skow Exactly. And the reporting thresholds are good examples. If you have a reporting threshold, does that mean there are not features that are below that? Of course they exist. But we're just like we don't care about them. They're not integrity threats, for example, but they exist. And if you're evaluating the measurement performance and you're trying to come up with a unity line, you do want to know that they're there, even though they have no value for the assessment of the integrity. This is the assessment of, just the performance of the tool itself. Now, if you cut that off, that's fine. However, the 45-degree line starts to get shallower because you have data missing. Just imagine you take the data away. Well now you have less data. So, you kind of have a shallower line. That's an effect that always happens. Now if you look at unity plots, you'll find a lot of them are not among the 45. They get tilted, they get shallower. And truncation is one of the key reasons for that. So, in this paper we were looking at if we have a truncated data set which is very common. And in this case, we had that, can we still do a level three with that data. And we checked a couple of ways of trying to solve this problem and came up with one that we thought was good. Rhett Dotson So, hold on. Wait. I just want to make sure I need to, because I'm chewing on what you said. These are not Jason. People say I talk fast. You talk quick. So, I'm like, I got to keep up with Jason here. Imagine our listeners are keeping up with you, too. Let's go. So, layman's terms again, we in a level one, we base or a level two, we based off a 45-degree line, right? 1 to 1 between ILI and the, the field. And even when we put the tolerance bands, if I can use that, those are at a 45 degree. What you're suggesting is that when we truncate the data, even for reasons that are perfectly acceptable, such as a reporting specification, we shouldn't expect those lines to sit at 45-degree angles, because we've essentially biased the data set. Maybe one way to say it or altered the data set. Jason Skow Yeah. And I will show that on paper. So, if you if you if you if you try to calculate the slope with, with the truncated data and just ignore that, you get a shallower slope. But now once you reintroduce the idea that, hey, we know there's data there, but it's just missing, now it gets closer to 45. Again, it's not necessarily exactly 45, but it gets much closer to 45, which is what we expect. Christopher de Leon And the idea behind the 45 for our listeners is it's the core component of unity, right? I mean, that's the idea, right? That what you're projecting is closer to what you were expecting. Right. Not within spec, but rather the idea of it's a unity plot for a reason. Right? You want the best alignment as possible represented by that 45 degree. Rhett Dotson So now I'm so curious, did you think of this on your own? Like, do you have an origin story for this? Were you on a boat and you saw the wake. Christopher de Leon Like he was he was ice fishing. He was ice fishing, and after your third shot of fireball, the light bulb went off. Rhett Dotson I don't think he drinks. Jason Skow Okay. I had a moment like you talked about, except it was not because I came up with the idea. It’s because I read a paper that showed that somebody else had this exact same problem, in a totally different context. And then they provided, you know, the mathematical model to solve it, which I use the same one. So, if you look at the paper, I, credit the original authors. For their methodology. Now, this was, the original authors are Houseman and Weiss are the two last names of the authors published. Christopher de Leon And so, Houseman and his wives? Rhett Dotson And I ask, what is wrong with you? Christopher de Leon Okay, I wasn't sure. Yeah, I'm trying to clarify. I just want to make sure everybody understood this so they could look it up. Jason Skow It wasn't just a smart, you know, a family of many wives, Houseman and Weiss. The two researchers working for the National Bureau of Economic Research in 1976. And what they did is, they were looking at an interesting problem, and they had the same problem as us, which is what kind of caught my eye. This was, like, super interesting. They were doing research on if they take, people in different, income categories and supplement their income so that they have sort of a constantly higher level of income, some of it's guaranteed, some of it they work for. They were wondering what the effect that would have on the number of hours that those people would work, like if you give people money basically guaranteed for free, not related to their work, do they end up working fewer hours? Right. And so, the problem was, you could not get into the study if your income was above some certain level. They were looking at the poverty level and they had different ratios or multiples of the poverty level, like 1.1, 1.2 times of poverty level. So right at the poverty level, a little bit more, a little bit more. But at some point, they would just cut it off and say, anybody make more of this. Can't be in our study. Like that's not what we're interested in. The problem with that is, is now once they were supplementing the incomes, some people's incomes, outside of the supplement would grow and some of them would grow less and they would grow into the zone where, you know, people that were cut out, cut out of the study they got truncated. They were truncated. Now, in this case, it was an upper truncation. So, if you had an income higher than some multiple of the poverty level, you could not apply for this program and you were not part of the study. And so, what the researchers were arguing, if we just do a best fit line to the data we have, we're missing all this upper income, population. We deliberately cut it out like it's there. We just did not include it. So how does that affect our study? And in their case, people before them that were doing similar studies were finding there was a very muted effect of this, income supplementation, meaning, you know, people were working the same amount of hours or close to, but they were saying, if your account for truncation now, it's actually a much more significant effect, meaning, people would actually curtail their number of hours if they were getting supplemented income. And, they thought that that was a more accurate, result. So, in any case, they had a completely different field of study, but they had the exact same mathematical problem that we have, which is we have a data set. We're trying to fit sort of a best line to it, but some of the data is missing. So how do we account for that missing part? They had a very clever methodology. Their clever methodology was based on, you know, looking at the scatter in the data. And then when there was a cut offline kind of assuming about what was down there based on the characterization of the rest of the plot. And so, they were kind of guessing, you know, it is a model, but it turned out in many cases that this is, a very accurate way, of trying to, estimate what's missing, and then you draw the best fit line through the real data and kind of what you estimate to be missing. And that best fit line is more on a 45. Rhett Dotson And so that that has you took, a practical application from another industry and brought it into ours and that that A is really cool. And has obviously implications on a level three. That's what that, that, that's what we're driving that. So, what are the implications if operators don't do this? It's that the slope of the line gets mixed up. What does that mean? Jason Skow Right. Well, if the slope of the line is shallower, usually the way it's plotted, you know, depends on what you choose for your x and y axes. But usually what it means is that the ILI is under calling the deeper features. So, I don't know if you guys have seen that in your experience. I've seen that many data sets that look at where you have an ILI tool and it's calling some depth, maximum depth. Then you go out to the field, and you find for the deeper ones, sometimes the field is deeper than what the ILI tool calls, which that's what that narrower or that shallower slope, is causing when in fact, if you're correct it, you'll find that it's more accurate. It is a more accurate measurement. Christopher de Leon Yeah. So, I just drew it out on a piece of paper. So basically, what you are saying because you have a flatter line, let's say your ILI tool was calling a 60% anomaly because the slope isn't steeper. It's shallower when you draw across the outcome of ILI possibilities, that band is wider. Is that so the range of features that it could have gets bigger. Whereas if your slope is higher, that range of ILI features gets a little bit tighter. That's clever Jason. Jason Skow And so, the, the kind of thesis behind this is, you want you have a data set. It's very expensive to get this data set. You want to use the data as best you can. You want to you want to squeeze every little bit of knowledge that you can out of that data. And so, when we talk about, you know, the different levels, you know, is it worth it doing level three. Well, in this case, if it's truncated and you're trying to get the maximum amount of knowledge out of that data, I think you know, accounting for truncation is one thing that gets you a little bit a little bit further down that line, you get a little extract, a little bit more knowledge, and in some cases, depending on what you're doing, that might be very useful. Christopher de Leon So, I'm going to ask you a question. So, does this mean I can do less digs and still achieve a level three? Come on Jason is this a cost savings opportunity. Rhett Dotson Why do this? How many days do I have to do Jason? Jason Skow Less takes than what? And the answer is, I mean, compared to a level three with a shallow slope. Yes. But compared to level two or level one, the answer is no. And sometimes maybe in some cases, but if you're if you're not, if you're not characterizing your ILI tool, well that will lead to more expensive, more digs that are quite expensive, each one of them expensive. It will lead to more. And if you have a better tolerance band, a narrow band and it's, you know, more on a 45 degree Christopher de Leon It also allows you to not have to target, potentially less injurious features, right? I mean, if we go back to thinking like, where do you normally get that field data, it's because I'm going after stuff that we think is an integrity concern, and that may have a certain morphology or a certain common characteristic. Right. It's above 50% deep or it's a 30 plus tolerance. So, your kind of might not have that lower threshold. Now you're not having to potentially waste resources do not waste, but allocate resource inefficiently, use inefficiently, use resources to go after less impactful integrity features if there's a numerical method applied to it. Right. Jason Skow And so, and that's why I love you, Chris. Right there. Yeah. Yeah. Rhett Dotson You know what I was going to say, that makes me so upset. I introduced this podcast talking about books and other applications and reading and that doesn't resonate with you. We're both reading economics, the mathematical models. Everything is for fun. You should love me. Jason Skow I wasn't as expressive about your, you know, initial tie to behavioral economics, but it really did impress me. So, I just I just didn't say it. Rhett Dotson It feels like you just said it after the fact. Oh, it's a cheap compliment. Jason Skow No, no, it's not. Christopher de Leon Did you have lunch yet? You're so spicy today. Jason Skow I have read a couple of economics books, they’re amazing. You know, they've kind of affected some of my thinking on, on similar topics. So, I was genuinely impressed. I just for some reason, I just, like, laid back. I didn't say anything, but it's true. Rhett Dotson Jason. I'm totally messing with you. Look, Jason, it’s been fun. I really want to say thanks for joining us. But before, as we wrap up and before we let you go, are you going to be at IPC this year? Jason Skow Absolutely. I'm going to be there. I'm helping with one of the tracks. I'll be running on one of the tracks. And I've got a couple of papers I'll be presenting as well. So yeah, hope to see you guys out there and, and any of the listeners as well. I hope to see them out there. Rhett Dotson Yeah, we will be there. I'd encourage our listeners, you get a chance to check out Jason's track, check out his papers, and check out this 2022 IPC paper, particularly if you're going to get into level threes. Chris, do you have any words of admiration or love you'd like to pass on? Jason, before we close out. Christopher de Leon Jason, we'll have to have you back on, buddy. Jason Skow Awesome, I'm up for it, guys. This was a lot of fun. Rhett Dotson Well, thank you so much for joining us, Jason. So, I want to say thanks to our audience for this episode, and we look forward to seeing you back in two weeks. Thanks to our guests and thanks to Jason. Appreciate it. We'll see you back. This episode of Pipeline Things was executive produced by Sarah Roberts. Thank you very much to our guest on the show, Jason Skow of Integral Engineering, our sponsors, D2 Integrity, and we'd like to thank the work lodge for giving us the space to record the episode.