The New Python Window in ArcGIS Pro

August 8, 2015 Leave a comment

Jason Humber:

New UI is great

Originally posted on ArcPy Café:

ArcGIS Pro is coming with a totally redesigned UI, and the Python window is no exception.

First off: the Python window is meant to be a utility. It’s not most flashy of designs. You type in code and get feedback, it should help you do that and that is all. Our intention in the redesign wasn’t to get people super excited about typing code into a text box, it was to pleasantly surprise you with some helpful features and get out of your way.

The way it’s set up it a little different in Pro App. Here’s a quick tour of its design and features.

Split window

There are separate places for input and output

pywin_pro_mage1

The biggest obvious change in the Python window is that it’s been divided into two pieces: the input section and the transcript section. We found from a usability point of view, mixing the input and…

View original 355 more words

Categories: GIS

Scheduling a script to run on a regular basis

August 8, 2015 Leave a comment

Jason Humber:

A good reminder that not all things have to be scripted

Originally posted on ArcPy Café:

This year at the user conference we had (once again) multiple user come by the island and ask if it’s possible to run a model or script on a regular basis, or at a prescribed time… the answer is yes. Click the link to the esri blog at Scheduling a Python script or model to run at a prescribed time for great detail how to do this.

Happy coding
-the arcpy team

View original

Categories: GIS

Dusty Blog

August 8, 2015 Leave a comment

Brushing dust off of the blog site. A lot can be said in 140 characters but it is not always enough to express your thoughts in rich detail.

Categories: GIS

Updated Links for arcpy Object Model Diagrams

December 29, 2014 3 comments

The links for the arcpy object model diagrams are dead and i was getting more and more comments/emails about it so I dug up the files and here are the links for arcpy 10.0 sp5, 10.1 and 10.1 sp1.

Categories: GIS

Python Add-in Assistant–Stubbed Out versus Build Out

This week we had some fun at the Esri International User Conference but still also made time to do some rock solid coding for Python Add-in’s.  One of the issues we were encountering was the repetitious and disconnected manner in which the Python Add-in Assistant works, well, not fair to blame the assistant, but more about how the XML remains static while the python addin module (your ***_addin.py file) goes through far more edits than the config.xml.  Take this simple example of a toolbar with a button and a combo box, begin with:

2013-07-12_17-32-11

and then add the toolbar, button and combo box:

2013-07-12_17-39-35

2013-07-12_17-39-41

2013-07-12_17-39-49

Click save an you’ll end up with stubbed classes for the button and the combo box in your ***_addin.py (menus_addin.py in this case) file and the information specified in the UI all sitting on your config.xml.  The content of which looks like:

<ESRI.Configuration xmlns="http://schemas.esri.com/Desktop/AddIns"
                    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Name>Stubbed Out Addin</Name>
  <AddInID>{b2d4e067-61f2-4919-8f82-95402e39763a}</AddInID>
  <Description>Stub</Description>
  <Version>0.1</Version>
  <Image/>
  <Author>Jason Humber</Author>
  <Company>Integrated Informatics Inc.</Company>
  <Date>07/12/2013</Date>
  <Targets>
    <Target name="Desktop" version="10.1"/>
  </Targets>
  <AddIn language="PYTHON" library="menus_addin.py" namespace="menus_addin">
    <ArcMap>
      <Commands>
        <Button caption="Spatial Abacus" category="Stubbed Out Addin"
                class="SpatialAbacusButton" id="menus_addin.btn_abacus"
                image="Images\abacus.png"
                message="Perform overlay functions via an Abacus"
                tip="Spatial Abacus">
          <Help heading="Spatial Abacus">Spatial Abacus</Help>
        </Button>
        <ComboBox caption="Choose Your Own Adventure"
                  category="Stubbed Out Addin" class="AdventureChooserCombo"
                  id="menus_addin.cmb_adventure" itemSizeString="WWWWWW"
                  message="Select-o-matic for Adventure Choosing" rows="4"
                  sizeString="WWWWWW" tip="Choose Your Own Adventure">
          <Help heading="Choose Your Own Adventure">Choose Your Own Adventure
          </Help>
        </ComboBox>
      </Commands>
      <Extensions></Extensions>
      <Toolbars>
        <Toolbar caption="Stub Example" category="Stubbed Out Addin"
                 id="menus_addin.tlb_stub" showInitially="true">
          <Items>
            <Button refID="menus_addin.btn_abacus"/>
            <ComboBox refID="menus_addin.cmb_adventure"/>
          </Items>
        </Toolbar>
      </Toolbars>
      <Menus></Menus>
    </ArcMap>
  </AddIn>
</ESRI.Configuration>

The XML is as XML often is, repetitious.

This is definitely not cool from a maintainability standpoint and given how touchy Python Add-ins to any miss type or typographic error we set about turn this stubbing out process into one that is more aligned with a build out process.

We’ve done a fair amount of XML processing (reading and writing using Python for working with things like XML Workspace Documents and configuration files and more so we decided to take this on in much the same light. Build ourselves a set of Python classes that understand their nested nature and know how to behave to serialize to XML.

Some of this can be automated using modules like generateDS but even that seems to be overly verbose and not always perfect in picking up the details on serialization. You can also rely on your IDE for XML manipulation (we use PyCharm and it has some really nice tools included for XML handling). No shortage of ways to get from XML to Python classes.

Now on to the use of the classes. Let’s write the python code that will serialize to the same as that stored in the config.xml file. In this particular case it would look like:

from os.path import join as osjoin, dirname
from pygp.core._addin import (
    ESRI_Configuration, AddIn, Targets, Target, Commands, Button, ComboBox,
    Toolbar, ArcMap, Toolbars)
from menus_addin import AdventureChooserCombo, SpatialAbacusButton

COMPANY = 'Integrated Informatics Inc.'
AUTHOR = 'Jason Humber'
NAMESPACE = 'menus_addin'
CATEGORY = 'Stubbed Out Addin'
DESCRIPTION = 'Stub'

abacus = Button(
    caption='Spatial Abacus', category=CATEGORY,
    class_=SpatialAbacusButton, id_='btn_abacus',
    image=r'Images\abacus.png', namespace=NAMESPACE,
    message='Perform overlay functions via an Abacus')
choosy = ComboBox(
    category=CATEGORY, namespace=NAMESPACE, class_=AdventureChooserCombo,
    id_='cmb_adventure', tip='Choose Your Own Adventure',
    message='Select-o-matic for Adventure Choosing')
toolbar = Toolbar(
    caption='Stub Example', category=CATEGORY, namespace=NAMESPACE,
    id_='tlb_stub', Items=[abacus, choosy])
commands = Commands(ComboBox=[choosy], Button=[abacus])
addin = AddIn(namespace=NAMESPACE,
              App=ArcMap(Commands=commands, Toolbars=Toolbars([toolbar])))
config = ESRI_Configuration(
    Name=CATEGORY, Version='0.1', Author=AUTHOR, Company=COMPANY,
    AddInID='{b2d4e067-61f2-4919-8f82-95402e39763a}',
    Image='', Description=DESCRIPTION, AddIn=addin,
    Targets=Targets(Target=Target(name='Desktop')))

Really? Just 32 lines of beautiful python, yup, that is all it takes (including imports!). Notice how native Python types are used for collections (i.e. lists) and the idea of passing objects around instead of strings for id and refID being embedded throughout the code is a very subtle but a very powerful addition.

In certain cases we have set some meaningful defaults on the classes so that we do not need to be as verbose. Then to generate the XML we just do something like:

path = osjoin(dirname(__file__), 'config.xml')
output = open(path, 'w')
config.export(output, level=0)
output.close()

This is of course a very simple example and we’ve taken this much much further by writing additional code that can inspect our classes and use that information to automatically populate the tool, button, menu, etc. Really awesome when you can use Python to manipulate Python to generate Python to run Python.

The results for those that are curious are posted below:

<ESRI.Configuration xmlns="http://schemas.esri.com/Desktop/AddIns">
  <Name>Stubbed Out Addin</Name>
  <AddInID>{b2d4e067-61f2-4919-8f82-95402e39763a}</AddInID>
  <Description>Stub</Description>
  <Version>0.1</Version>
  <Image/>
  <Author>Jason Humber</Author>
  <Company>Integrated Informatics Inc.</Company>
  <Date>7/12/2013</Date>
  <Targets>
    <Target version="10.1" name="Desktop"/>
  </Targets>
  <AddIn namespace="menus_addin" library="menus_addin.py" language="PYTHON">
    <ArcMap>
      <Commands>
        <ComboBox category="Stubbed Out Addin" itemSizeString="WWWWWWWWWWWW"
                  tip="Choose Your Own Adventure" sizeString="WWWWWWWWWWWW"
                  id="menus_addin.cmb_adventure" caption="" rows="4"
                  message="Select-o-matic for Adventure Choosing"
                  class="AdventureChooserCombo"
                  refID="menus_addin.cmb_adventure">
          <Help heading=""/>
        </ComboBox>
        <Button category="Stubbed Out Addin" image="Images\abacus.png"
                tip="Spatial Abacus" id="menus_addin.btn_abacus"
                caption="Spatial Abacus"
                message="Perform overlay functions via an Abacus"
                class="SpatialAbacusButton" refID="menus_addin.btn_abacus">
          <Help heading="Spatial Abacus">Spatial Abacus</Help>
        </Button>
      </Commands>
      <Toolbars>
        <Toolbar category="Stubbed Out Addin" caption="Stub Example"
                 id="menus_addin.tlb_stub" showInitially="true">
          <Items>
            <Button refID="menus_addin.btn_abacus"/>
            <ComboBox refID="menus_addin.cmb_adventure"/>
          </Items>
        </Toolbar>
      </Toolbars>
    </ArcMap>
  </AddIn>
</ESRI.Configuration>

Categories: ArcGIS, GIS, Python Tags: , , ,

Documentation not Optional

Now that we are auto-generating our binary toolboxes fully from code we have also been able to introduce a really nice build process that generates our “calling code” scripts, tool validator code, and then validates all of our tool code (classes and functions), tool parameters, stylesheets, referenced/supporting files used for defaults, and, gasp, documentation too at the parameter and tool level. 

Some might think it overly involved to have a build process for script tools but having been building toolbox tools since the inception of this capability, back when ArcToolbox was a stand-alone application (shout out to anyone else who was building tools then!), this should just be an obvious evolution.  It might not make a lot of sense for a throw-away tool or just building one or two tools but having built, well, probably more tools than come with the core of ArcGIS over the years this again just makes good sense.

While the ArcGIS product continues to evolve and improve there have been some portions that have degraded, in particular, the documentation tools for documenting tools.  In 8.x this was all done in code so it was up to the developer – too bad developers generally write in API speak and in terms of nuts and bolts, not in terns of usage or usability.  Issues in 9.x days where if you changed the name of a parameter your documentation got dropped, no warning, no message, it just got dropped.  Fast forward to 10.x, what do we have? Documentation for tools is handled the same as editing metadata on a feature class.  Not sure about that – what is the extent of a tool? and what makes sense as a thumbnail?  It just seems forced and the new editor is functionally inferior to that from 9.x days.

We decided that we needed a better way, a way that allowed our developers and analysts and specialists to all work in the same manner and on the same platform and contribute to better documentation.  In our case we adopted reStructuredText as our standard for documentation creation (mark down is fast to write and works really well with source control).  We had actually adopted it several years ago for our Python package documentation standard for parameters and also as part of a movement towards separating content from presentation for other types of technical documentation and for generating conditional documentation.  The same technology choice being used in three or four different ways means we get to become proficient and much more comfortable in that technology and learnings become universally applied.

In our build process for tools and toolboxes now we perform the necessary conversions from mark-down into mark-up compatible with binary toolboxes.  Yes, it does mean needing to dive into dark areas of ArcGIS that probably were not intended to be used in this way but the rewards are great and the time saved phenomenal and the consistency across tools is outstanding – mostly due to the way in which documentation can be reused without copying – and leads to greatly enhanced usability.

The functionality for generating toolboxes, tools, and their documentation is included in our pygp package.

But Binary Better

After working through the idiosyncrasies of python toolboxes over the last few months we’ve found ourselves saying that python toolboxes are good but binary toolboxes are better. 

In our trials we did come up with some really nice patterns and processes for use with python toolboxes. Examples include: lazy imports with correct import context to avoid compilation of modules when ArcGIS loads, auto generated documentation (more on this in another post), ability to use .net executables with messaging in a python toolbox, and a half dozen or so other items that are now included in pygp.

One thing worth praising of python toolboxes is that the parameter class got some better smarts. We decided that we wild build around this and ended up developing a set of patterns for use to build script tools. So what? Well it is pretty significant because now from Python we can actually automatically build our binary toolboxes including the script tools and all associated documentation and this includes full handling for tool validators.

The whole process is pretty awesome and is saving us hours and hours of effort especially when working in multiple computing environments (dev/test/prod).

Python code building script tools in binary toolboxes. This is what python toolboxes should have been.

Categories: ArcGIS, GIS, Python Tags: , ,
Follow

Get every new post delivered to your Inbox.