I chew gum, a lot of it. This morning when unwrapping a package I saw a little icon on the package of a man putting trash into a garbage receptacle with the slogan “Do Your Part!” alongside. Seems like a noble slogan…but that’s not the full story….
What was the lineage of that gum packet? It began life inside of a bulk package so by the time I saw that message I had a) unwrapped plastic from the box b) opened the box c) removed the plastic from a bag containing three packs of gum d) removed the plastic from the individual package…then to fully enjoy that pack of gum I had to unwrap the gum. That makes 6 levels of packaging…something like a modern day matryoshka doll.
Maybe this gum manufacturer should point the finger back at themselves, I’m glad to do my part, just as long as they tow the line as well.
Yet another day of traveling (YADOT). In life the journey is usually as important as the destination, but there’s nothing glam about being cattle prodded into a sealed tube and transported to a foreign country while being forced to watch mind numbing kiddie movies.
Lately I travel with a new card, more than anything else it shows that I’m a member of the road warrior elite, it bears my name, and yet another picture on government documentation that shows me as being way too tired, either that or caught in the middle of blink…in either case my eyes are only partly open. Which is funny, because now that I can get my irises scanned as a means of going cross-border it is humorous that only someone who travels too much and is jet lagged most of the time would find the iris scanning of benefit, too bad it requires that you can actually open your eyes for more than a few moments at a time. Hey, maybe that’s just a challenge for me, a weird combination of hating direct natural light (unless accompanied by a beach or a pool) and buzzing on too many espressos has made posing for the lady in red an adventure.
So why from random import choice? Well today I stood in line and was randomly chosen for secondary screening, no harm no foul, and then the next guy behind me in line was randomly chosen too. Hmmm, doesn’t seem as random when that happens. All good though, no rights or orifices violated. Still in possession of my luggage and, at least for this trip, my dignity.
In our Segregating Foreign Keys post we started talking about using database metadata to identify constraints. We made a distinction between foreign keys that are implemented for lookup values and those that are purely for referential integrity. In this installment we’ll look at how we can translate the lookup tables into something useful in a geodatabase. While we’re at it, we should also bring back into consideration the check constraints we played with in our Constraints and Comprehension post.
The first translation is one of terminology, lookup tables in a geodatabase design are handled through domains, and more specifically, as coded domains. Coded domains are also the poison of choice for check constrains (or at least those that are implemented with IN statements).
The next step or at least what I’ve got in mind here (and I guess by the end of this post we’ll know if it works or not) is to write some simple Python code to reformat the metadata we retrieve from the database and place this in another structure that can be used by the built-in tools of the ESRI geoprocessing framework for creating domains. But only creating the domains is pretty useless unless we also assign the domain to a table (unfortunately the random order of my posts means that we haven’t handled tables or feature classes yet so we’ll have to do the assignment of domains once we get around to handling the migration of relational tables into geodatabase tables and feature classes).
The geoprocessing tool that we plan to use is TableToDomain in the Data Management Toolbox and the parameters for using that tool are in_table, code_field, description_field, in_workspace, domain_name, domain_description, and update_option. So all this means is that we need to get the lookup tables and check constraints into a table structure and ESRI will handle the rest of the magic for us. Pausing for a second, have I mentioned lately how coolness and value of the ESRI geoprocessing framework? Definitely one of the best things to come with the 9.x release(s).
Assuming that we’re starting with the resultsFilter defined in the Segregating Foreign Keys post, the following code snippet allows us to put the lookup tables into a domain (note: targetWorkspace is undefined in the snippet but represents the output geodatabase).
for r in resultsFilter:
#Query the database
cursor.execute(‘SELECT %s CODE, LONG_NAME VALUE FROM %s’%(r[6], r[5]))
#Fetch the results and the record set definition
codedValues = cursor.fetchall()
cursDescription = cursor.description()
#Make a new table
tmpTable = gp.CreateTable_management(‘IN_MEMORY’, gp.CreateUniqueName(‘iii’, ‘IN_MEMORY’))
#Add fields — precanned helper function
iii.addFieldsCX(tmpTable, cursDescription)
#Now insert the coded values into the table
iRows = gp.InsertCursor(tmpTable)
for cv in codedValues:
iRow = iRows.NewRow()
[iRow.SetValue(f[0], v) for f, v in zip(cursDescription, cv)]
iRows.InsertRow(iRow)
del iRows, iRow
gp.TableToDomain_management(tmpTable, r[6], r[5], targetWorkspace, r[0], r[0], ‘REPLACE’)
A similar approach can be applied to the check constraints, but we’ll leave this as an exercise to the reader…
What is it about a blog that make it a good blog? Some would say that frequent updates are important in order to keep readers coming back, others would argue that it is about content, yet others would focus on the levity and loftiness of concept.
Several of the GIS blogs out there are really good and I visit them on a irregular and in frequent basis to read about the author’s latest trip to a conference or to view a regurgitation of press releases and, my favorite, pointers to each other’s blogs.
For me the blogs that catch my attention are those that are solid references of information that is not found elsewhere. People willing to share their experiences for the sake of allowing others to gain insights and for improving the community — not in the sense of volunteering at the local homeless shelter — but more about willing to “put it out there” to have “it” adopted or challenged.
By the way, if any of you get a chance, check out the book Crowdsourcing, awesome book, great commentary on open source movement and the interaction within the communities that fuel most of the innovation present online and beyond (aka real world).
The professionals most likely to use a decision support system will expect the jargon of the system to coincide with their understanding and workflow. Linking decision support systems back to those who plan to use them is an understated yet invaluable aspect of the design.
Why such a preamble to this section? Well if you look at the body of knowledge that exists (i.e. books) for pipeline costing you will see that the authoritative resources are those based on grasping the complexity and dynamics of construction and even though technology is a large part of the overall construction activity, the dominant factors still reside with the personnel and their tools/equipment. What is more dynamic than people?
Materials and construction are two major constituent components of cost with construction costs outweighing material cost about 2:1 and whereas materials are highly related to pipeline length, construction is much less reliant on pipeline length. Easy to see where the focus should be placed in order to reduce the overall cost!
Looking to the literature [insert reference to cost estimating book] we see that construction labor costs are affected by productivity. Shock! This shouldn’t come as any surprise to anyone who has worked in any industry that better productivity results in lower labor costs for the same end-product. Why then do we not see this thought process applied more readily to pipeline routing? Probably because the linkage of all the parameters required to perform such a routing effort is difficult – but then again, since when has something being difficult been a good reason not to do what is right?
One major consideration when using an effort and productivity approach is to separate dollars from the mix. Dollars will change wildly based on the country in which the construction is happening so applying after the fact on top of the overall productivity and effort is the better approach.
Labor Productivity
According to [inset author name] the main components to be considered in establishing cost estimating factors for productivity are: General Economy, Project Supervision, Labor Relations (workforce), Job Conditions, Equipment, and Weather.
Each of these factors is considered equal in the overall weighting of productivity factors and although the math is easy, establishing the factors, and getting buy-in from your stakeholders, is the real task. Summarizing from [insert author name], let’s look at how to assess each of these factors on an individual basis. (Easy enough to state but determining the relative rankings of these factors
General Economy
Considering the General Economy means having to assess the state of the region where the project will be constructed and will require some knowledge of the local business trends and outlooks, the employment situation, and the construction volume.
One might think that if the general economy is in good shape that this makes it better for the project, but no, it actually works against you! Why? Well it should be obvious now that you’ve had to stop for a few seconds to think. When times are good, most everyone is busy regardless of their quality of skills/craftsmanship and the ones that remain available to the workforce are those that are not good enough to work. Ouch, this basically means you are pulling from the bottom of the barrel for personnel which is never a good thing when you need high quality (or at least some quality) and focus on a job well done.
Conversely, think of the motivation to your workforce when they know that they are highly expendable. Not that you want to fear people into doing what is right, there is a certain passion that gets sparked when you know you’re hanging onto a job that five other people are just salivating to take from you.
The condition of the economy in your region of interest will have an affect on your overall productivity and, although weighted equally with other factors, should be given the most careful of consideration. Efficiency of the workforce increases as the state of the economy decreases – survival instinct.
Project Supervision
Can you thrive under poor management? Some people can but when you’re building the large project you need all people to thrive.
In construction terms, you need solid supervisors and foremen (forepeople? Perhaps there is no politically correct way to state this one) to achieve your productivity goals. Quality of supervisory staff will be related to some hard and soft qualifications such as experience, availability, salary, technical knowledge, people skills, and leadership.
Not to give the impression that we’re trying to find the next CEO but you can never overstate the importance of a positive team dynamic, especially one where the team members are respectful of their leader.
Workforce
Not to belabor (oh, that was a subtle pun) the point of having good people but much the same as having good supervision, you must also have a good workforce and let’s define good as available, experienced, and having good skills.
Job Conditions
Even if you have the best workforce possible available at your disposal you’ll find it hard to attract and maintain such folks if the project has an air of failure or discomfort.
Although the geography of the project may not be entirely under your control (see study area discussions) a defined scope and schedule are certainly within your control. Main considerations here are scope, schedule, material procurement, and ease of operations.
Equipment
Good people cannot do it alone and will need to have access to the proper tools and equipment. Questions to ask when assessing this part of the equation include availability, condition, maintainability, and usability.
Weather
Not too many pipelines (if any) are ever built entirely indoors. What? Thanks for stating the obvious? The counter to this idea is that construction happens in the great out of doors (doesn’t it just sound presumptuous to write it that way instead of just ‘outdoors’) where weather and climate are real and relevant considerations.
Not every vocation is as committed as the postal service “Neither Rain, Nor Sleet, Nor Gloom of Night” in delivering the goods so we need to put a realistic slant on how [sorry to all those meteorologist for the definitions that follow but I feel it only fair for you never getting my local weather predicted properly] climate (characteristic conditions) and weather (day to day conditions) affect the ability and availability of the workforce to perform. Why ability and availability? Simple, there are rules in place to protect your workforce from harsh conditions and while some conditions require massive amounts of downtime (imagine how having 25% downtime for breaks affects labor productivity) and even if the breaks are missed, skipped, or it is just miserable outside and there is no guidance on how to handle the scenario, you can just imagine that the worse the conditions to more prone you are to mistakes (mental or physical) which will always manifest themselves in the end during quality assurance (hopefully) or during commissioning.
Construction Effort
It might seem like a copout (and perhaps it is) but we’re not going to address the construction effort factors at all except by way of example in future installments and to state that these are really important so please remember to engage those that know. Reason for omission? We have a few ways by which we establish these factors (and our clients have some excellent approaches too) but we really don’t feel like giving it all away.
We should probably look at some basic accounting such as fixed and variable costs. Not so much that we want to be experts in economics but we at least need recognize some really basic elements of business. Easiest way to explain many of these economic factors is through way of example so let’s pretend that we are in the world of manufacturing computer widgets.
Fixed Cost
Fixed costs must be paid even regardless of the amount of production at our widget factory, even if the factory does not even produce one widget. How do these costs manifest themselves? These costs are things like mortgage or rent and taxes for the building and the land on which it stands, they also include the expense incurred for having equipment, machinery, tools.
These costs are typically paid on a regular basis, for instance monthly, and may rise or lower over time due increases in valuation or decreases in equipment costs once the initial principal is paid off but there is no linkage between the fixed costs and the number of widgets produced (assuming the factory stays within its initial design capacity).
Understanding Fixed Costs for Linear Assets
Standby and Equipment
Examples of fixed costs in linear asset decision support systems are the non-productive costs associated with down time during construction, for example, standby time for a crew due to weather, poor working conditions, lack of supplies, etc.
Just like downtime for people, equipment also has a cost of ownership or rental – directly analogous to our widget factory, even if we aren’t constructing the asset we still need to pay the bills of the equipment in order to keep the equipment allocated to our project.
Junctions
Do we need to handle fixed costs in our decision support systems? Most certainly! But what do these look like when we’re trying to site a linear asset? Might not be obvious at this point in time but the junctions (i.e. interchanges, pump stations, poles, etc.) along a linear asset are examples of fixed costs. How can that be? More junctions are required as you increase the length of the asset, wouldn’t this indicate that we really have a variable cost?
In some cases where junction density is high this argument could be made but this argument cannot be made in general. Take for example a new interstate highway being developed between two cities, clearly you need an on/off ramp at each end but you don’t really need any other entry or exit points anywhere else along the route. Might not be the most effective way to plan a transportation network but there are plenty of real-world examples of this arrangement.
Take this same example a little further; did we even mention how far apart these cities are from one another? No, we could build a 10km highway, 100 km highway and still the same number of junctions.
Now change the example and look at something like pump stations along a pipeline, this must be tied to length right? We yes it is tied to length but this still does not mean our argument fails. Use the example of a gravity drain slurry pipeline for a mining operation, no pump station required! Or another example of a liquid pipeline through rough and undulating terrain, the number of pump stations will be a function of hydraulics and dominated by changes in elevation.
Variable Cost
Variable costs, on the other hand, are linked directly to the number of widgets produced and can be thought of as the costs of consumption. For example, variable costs include raw materials and utility fees for running machinery and equipment.
Understanding Variable Costs for Linear Assets
What is the unit?
The unit of measure in our widget example is the number of widgets – easy enough, but what is the unit we use for linear assets? This should be obvious to the reader but let’s not gloss over the important stuff, the unit of variability in linear assets is distance.
How do we measure distance?
After many years of working with geographic information systems and decision support systems I am still amazed at how simple it is to ask a question having a complex answer. “How do we measure distance?” is one of those questions and we certainly do not want to address all aspects of this question here (the literature is already full of some meaty reviews on distance).
The main thing we want to point out here is that depending on your need for realism and your desire to properly model the phenomenon surrounding your linear asset you will need to be able to handle multiple distances simultaneously. Simultaneous handling of costs since some costs scale by two-dimensional distance, some scale by three-dimensional distance, others scale by effective distance, and yet others scale by complex functions of distance.
Regardless of the approach taken with respect to cost units there will always be some level of constancy and variability in the costs. This aspect of cost modeling is one we see being ignored in most implementations and the only attempt we’ve seen to try to address this issue (other that our implementation of course!) has been as a post processing exercise. This post processing approach is certainly valid in that it produces a result but it misses the mark by trying to fix an artifact of the implementation instead of fixing the actual implementation.
In a future article we’ll describe and contrast the affects of ignoring cost components, including cost components, and post processing. Delicious ambiguity!
