Filed under: Python
Anyone working with schemes for Python in Komodo would likely have seen the keywords2 entry in the Element Type (Edit Preferences>Fonts and Colors>Lang-Specific) drop down. For the longest time I was wondering what was in this list of keywords so I spent some time on the ActiveState forums to see what could be found.
All I was able to unearth was a short post from the development team mentioning the koPythonLanguage.py file that lives under the installation folder. Poking around a bit I found that by default there are no values in keyword2 (note that in another post I see that keywords2 might be deprecated, sure hope not it is quite useful!).
The post didn’t specifically say what needed to be changed but I was able to find some similarities in the other Python files living in the same folder. I should mention that my motivation here was to get some specific keywords highlighted, well, maybe they aren’t real keywords but they are “by convention” keywords and it is really handy to be able to see these more clearly when working.
Cutting to the chase, to make the keywords2 element type behave as expected (aka do something) then all you need to do is add a line into the KoPythonLexerLanguageService class (line 75 in the screen capture below).
As you can see the self and cls keywords have been added and now I’m able to add highlighting to these. Another nice thing in the koPythonLanguage.py file is the Python Sample Text, I’ve updated mine so that I have decorator and number examples.
I remember back to a VB.NET course I took quite some time ago, the instructor made a comment in the first class to the effect that “all real applications use a database”. That wasn’t a surprising comment and it actually reflects most of the projects we take on, although I’ll have to stick with a classic definition of database to mean more than just relational.
With a few years under our belts using Python for scheduling “things that go bump in the night” type processes we’ve seen our fair share of weirdness on the back-end setups of client shops and (this may come as no surprise) an oddly high amount of downtime from data providers (sans outages notices might I add).
Most of our hard core data manipulation programs utilize some type of read or write from a proprietary or public data model (implemented in a pure form or with extensions). In any case, we end up using the Python DB-API modules that are made available online, a variety are available and depend upon the underlying database engine. If you read through the Python DB-API specification you’ll see what is indeed mandatory and what other syntax sugar can be added. There are some modules out there that have a plethora of additional capabilities (most of which works!) and I’ll give them credit for not stomping to hard on the DB-API.
We spend much of our time working with ESRI specific databases and data formats and with their commitment to Python over the next few major releases of ArcGIS we’d sure like to see them become more standardized in their Python approaches, for example, up until 9.3 it was tough to consider anything Pythonic in their implementation. Now with 9.3 we see that they are taking it serious and working to improve their use of POPOs. One step further (and this should be relatively straight forward for the development team at ESRI) would be to implement a proper DB-API that at a minimum implements the required components of the DB-API and then adds spatial capability as well.
For those that are interested we’ve already been working on a Python module to allow such “expected” capability. Most of the individual components are written and next steps are to collate the components into the expected DB-API form.
Filed under: General
If it weren’t for plane rides I ‘d probably cancel most of my magazine subscriptions, no shortage of time spent in the air these days and the solitude of air travel, as odd as that it may sound, certainly does give me the opportunity to read.
This month in Alberta Venture they had an interesting article “Taxes and Duties”, the topic of taxes is never that exciting but the stance of this article covered definitely spanned some interesting ground. Janet Keeping’s comments are brilliant and refreshing, so good to see that there are people in the tax world that are above-board and actually understand the value of taxes.
My opinion on the topic is simple and ideal — looking for loopholes to avoid taxes is a poor substitute for contributions to the community either directly or via taxes. Greed is not the same as ambition.
In decision support for linear assets the most familiar result is the optimal alignment but this is far from the only useful output from a rather rigorous process. In many respects, the optimal alignment is more misleading than useful since it only provides a glimpse of what is without painting a full picture of the situation – like reading an executive summary without regard for the underlying body of knowledge. What then in the decision making process can provide us with additional insights? Cost corridors!
Before we plow into talking about cost corridors let’s remind ourselves that least cost analysis is merely a numerical technique for solving a complex system. With this now fresh in our minds we can talk a little more abstractly, numerical techniques used in solving systems operate in the realm of a solution space. Solution space is not constrained to two, three, or four dimensions but rather the dimensions are as numerous as the number of variables, constraints, and interaction rules that comprise the system.
As mentioned, the most common or expected result of a least cost analysis is the optimal path, this represents just the minimum solution, and, for those who have studied approaches for solving systems, we know that this is just one solution found within the solution set. Observing the solution set is where cost corridors come into play. The unfortunate thing is that even though solution space can handle multi-dimensions we mere mortals (okay, more to do with technical limitations) need to determine the projection of the solution set onto the practical plane of a GIS graphical display. The cost corridor is the intersection of the solution set with combined cost surface.
In earlier posts we detailed how the least cost analysis process works when all that is needed is the optimal route, quite simply:
- establish a start point
- accumulate cost weighted distance from the start
- establish an end point, and
- trace the steepest path from the end point to the start point.
The process for creating cost corridors capitalizes on the optimal route process quite heavily and is implemented as:
- establish a start point
- accumulate cost weighted distance from the start
- establish an end point
- accumulate cost weighted distance from the end
- combine start and end accumulated cost weighted distance
- find the minimum value within the combined accumulated cost weighted distance, and
- using a cutoff percentage query the combined accumulated cost weighted distance for values between the cutoff and the minimum.
Check out this for some more information on this process and its usefulness.
Filed under: Python
Another great get away to recharge and read and read and read and read and … yup, always lots to stay on top of in our ever changing dynamic world. Had some real fun taking in the “new” features of C# 3.0 and the .NET framework 3.5, some really nice things included there like LINQ and lambda but other than that, just more CLR chatter and hype around how the new VS makes coding sooo much easier. Probably a good thing considering how user expectations continue to heighten more rapidly than quality code can be created and pushed into the cloud.
Most would know that I’m into Python fairly heavy so I figured to bridge some of the “new” .NET features like LINQ and lambda back into their corresponding cohorts inside Python. Although not exactly the same thing, it does for my purposes cover off the majority of items that we handle in our code development, the LINQ as is pertains to SQL is YAORM (yet another object relational modeler). In Python we’ve had for quite some time a set of wonderful modules that provide this exact capability, granted they are not part of the core development but none the less they are beneficial packages if you’re doing anything more than one or two simple SQL statements. The modules include some favorites like SQL Object and SQLAlchemy – if you like the idea of being efficient for your clients then you want to look at these and some of the plug-ins too.
Onto lamdba’s, well this isn’t a new thing for Python peeps, been around forever. Of course you always need to ask yourself “is there ever a chance that this function will be used elsewhere?” which normally equates to “Yes” but if not then lambda away! Here is one of my favorite ways to use a lambda, it is in the key kwarg for sort or sorted. This approach I find really handy for being able to capture the sorting logic directly inline in the code which for me really helps in readability!
import random
import operator
lowerBound, upperBound = 0, 1000
values = [tuple([random.randint(lowerBound, upperBound) for i in range(10)]) for i in range(1000)]
sorted(values, key=lambda v: sum(v[:2])**2 + sum(v[2:5]) + reduce(operator.mul, v[5:], 1.))
What this code does is generate a list of tuples containing values and then the sort is based on a combination (like a weighting) of the tuple components. Way cool and super fast.
Well enough on that for now, nice to see that C# is beginning to catch-up with where Python has been for quite some time…YATTOS (Yet another testament to open source).
Filed under: Python
Recently we finished off a very aggressive sprint to release a new set of tools for one of our clients. Much fun had and the delivery and roll-out went very smooth – great teamwork all around and excellent interaction with the client internal IT department.
One item that crept up late in the development cycle (not that we didn’t know about it, just that we hadn’t thought through the full set of implications) was localization. This really hadn’t been an issue in previous releases but with adding new functionality and capability related to dates and times, well, of course it was now an issue.
The application needed to work regardless of the personal preferences each person had set for their date/time format. Hmmm, will that require a bunch of string manipulation and testing and checks, or maybe, just maybe the good folks behind the Python language have already crossed that bridge…and, of course, they had.
A word that comes up often when talking about Python and implementation is elegance, and this time was no exception. Simple solution to what could have been a nightmare coding effort in the final days. Hanging off of the datetime (and time) modules is strftime and one of its format strings does exactly what we needed, for example, the following interactive code shows how a datetime is return in my system (local) format – way cool!
>> import datetime
>> t = datetime.datetime.today()
>> print t
2009-03-26 08:25:29.160000
>> t.strftime(’%x’)
‘03/26/09′
Where this really becomes a benefit is when working with an ESRI cursor object, and you’re wondering “How the heck should my date be specified so that it ends up in the database correctly?”, next time just wrap it with some %x and let the magic happen.
Filed under: GIS
I’ve been looking lately at Bluetooth technology, you know, how it works, what makes sense (i.e. not real time sound) for its use, range and a few other things. What got me on this topic is the seemingly small selection of Bluetooth mouse and keyboards — the computer peripherals that should be the mainstay of Bluetooth products.
In all that searching I have found a nice headset from Plantronics Pulsar 590A, and another from Motorola S9-HD — both are rechargeable and the range on the Plantronics is phenomenal. Favorite mouse has got to be the Logitech 470 with a close second being the Microsoft Notebook Mouse 5000 — battery life seems to be a tad bit better with the 470. Finally for keyboard/mouse combinations the Logitech DiNovo is slick, and the Microsoft Wireless Entertainment Desktop 7000/8000 are nice options too (although the mouse is a little heavy in the Microsoft bundle).
Not a Bluetooth keyboard, but check out this new one I picked up today today.
Filed under: GIS
Continuing on from a post back in January where we talked about a really nice database engine for road warrior development. The question, when posed to our internal development team, was “what about spatial?” — a very relevant question given the line of work we are in.
So in my travels (this time just meaning a cruise around the web), I stumbled on a really cool set of open source code that extends SQLite with spatial capabilities. The package is called SpatiaLite (http://www.gaia-gis.it/spatialite/) and offers up the very useful blend of SQLite relational database and GEOS library spatial operations, the result is a lightweight spatial database with built-in spatial operators.
If you check out the website you’ll also see something else that is fairly exciting – it’s called VirtualShape and it is a driver for SQLite that enables SQLite databases to access the data held in shapefiles. Wow! This might be the closest thing we’ve had to the live spatial operations we could easily perform using Avenue and theme definitions (and if you can remember writing something like [shape].contains(…) then please drop me an email).
Filed under: Python
Kudos to anyone who has tried to code on plane or in an airport without noise canceling earphones. Not only is it way too noisy but the lighting can really mess with your eyes after a short time. My main issue when trying to accomplish something while in transit is finding enough space to open up my laptop fully, which means my viewing angle is far less than optimal.
On my last journey to the south I took some time to read up on colors and fonts used in IDE’s. There are a bunch of good sites out there with solid suggestions on programming fonts. For me I initially used Courier New (come on, who hasn’t accepted the defaults at one time or another) but soon switched over to Consolas when changed over to Vista — enjoyed Consolas for a while but then fell in love with Bitstream Vera Sans Mono after a stint working on one of my Ubuntu laptops.
For most (but not all) of our Python development we use Komodo IDE, this IDE allows full control (although the granularity of control could be better with respect to the categories of language Element Types) over the fonts and colors used. Here is the first scheme that we used (and some still use) in our shop when developing:
This one worked for quite some time for me but it was too harsh on my eyes when working on my laptop screen (using 1920×1200 resolution) but was OK when on my external display (24" Widescreen LCD). Fiddling with the various settings I’ve come up with these two schemes that really help reduce eye strain for me on my laptop screen and external display, even when surrounding light and viewing angles are sub-optimal.
This last one is the scheme that I’m presently using, it seems to work great on the plane — a subtle change is that the operators are bold, subtle but makes a difference.
Filed under: GIS
A new year and some new eats, trying to cut out those heavy snacks that make you feel bloated — remarkably similar feeling as working with some enterprise level databases that are poorly tuned…At one point in my career it was uber important for to be on top of all the latest and greatest technology available; that was in the days when web-time was being defined and it certainly made for some interesting reading, but the relevance was soon lost — the world of business does not move at the speed of technology change and rightly so, change is expensive (but can also be a good investment when properly handled).
Even though we have fully wired environment and hooked-in computing environment that allows us to work remotely, the convenience of this is soon lost when you spend a lot of time sitting on planes or are unable to get a reliable signal that’s strong enough to handle the data throughput. For example, working a pet project a few weeks back while on the road saw that nearly 95% of the time spent waiting was for data transmission across the skinny wire 3000km away.
This got me looking for an alternative…something that would be lightweight, easy to code against, worked with Python, and for which the code was easily transferable (i.e. SQL DML and DDL are very rigid to ANSI-SQL) by only needing to switch a connect string.
First path was to look at an ORM, SQLAlchemy is not a database engine but it does offer the convenience of providing a layer of abstraction between your code and the database. Although SQLAlchemy is an awesome package, it was overkill for what was really needed. Next thing, was looking at SQLite — wow, what a slick database engine and does it work with Python, for sure, built-in for 2.5 and available for 2.4. Granted SQLite does not support all the capabilities of Oracle or SQL Server, it is remarkable close in its syntax and capabilities and far better than MS Access with respect to syntax and has a very tiny memory footprint.
