Planet Perrygeo

In February, at the Toronto Code Sprint, the PostGIS team looked each other in the eye (for the first time) and committed to get the 1.4 release out by late April.



Well, it's late June now. It seems very likely that I will get to cut 1.4.0RC1 tomorrow morning.



My personal preference has always been to release early and often. In the hacker ethic, this sounds like a good thing, it's the "bazaar" model that Eric Raymond promoted over the "cathedral" model of development. In the bazaar, you dump out regular releases, and let the community dictate whether they are of quality ("don't use 2.31.2a, it's garbage!"). I still remember being told by a more knowledgeable Linux user that I could upgrade to 1.1.53 (?), but not any further than that, because the succeeding releases were unstable. In the cathedral, you release no wine before its time, aiming for a polished diamond of a release.



So, 1.4.0 has taken much longer than expected, the confluence of a development team that is now unwilling to accept the existence of any "crasher" bugs at all (no matter how unlikely they are to be exercised) and a growing comprehensiveness in the test suite, which is now covering all the functions, in most every combination of inputs. Because of the enhanced testing, we discovered crashers we didn't know we had – and then we had to fix them.



Despite chafing to release! release! release! I have come to appreciate our new conservatism. Among my favorite feedbacks on PostGIS is the users who say "it just works, install it and forget about it, rock solid". That feels good, and to keep things that way, our new austerity is only going to help.



The maturation of PostGIS into a product you can just "install and forget" has been multi-stage.



Prior to the 1.0 release, Sandro Santilli added the first regression tests. These tests have been growing ever since and have been invaluable in ensuring that old bugs don't re-enter the code base, and that new features don't break old features.



For the 1.4 release, the documentation was upgraded substantially, by adding a great deal of extra structuring to the reference section. Regina Obe discovered that a side effect of the extra structure was that she could automatically generate a test for most every documented function using XSLT on the docbook XML. This new "garden test" found a number of previously undetected bugs, that have since been removed.



For the 1.4 release, I added the start of a CUnit test suite that exercises the PostGIS functions without requiring a database back-end. Even in it's early state, it has saved me from a couple booboos already. For future releases, this extra regression suite is going to help keep things stable.



For the 1.4 release, Mark Cave-Ayland re-worked the logging and debugging infrastructure, to make the coding cleaner and easier to maintain during debugging cycles. He also split out the underlying geometry implementations, which are now used in the loader/dumper utilities, for a more consistent approach to geometry handling.



These are all under-the-covers improvements that end-users never see. But they all contribute to that "it just works, it just runs" end-user experience that I have come to treasure even more than the sensation of slamming out a point release at 2am. I hope everyone tries out RC1 so that we can slay any remaining bugs before the 1.4.0 release!

 

"Green shoots..." ah, for the good old days of only two weeks ago, when green shoots were in our future...







I never really understood why decreases in the rate of change of unemployment were considered such great news. "Good news, the second derivative has gone positive! we're plunging into the abyss slightly less quickly!" Only in a world of rampant, congenital optimism – or statistics-induced myopia – could four months in which 18,300 Americans lost their jobs every day be described as a period of "improving conditions".

 

There is a common online question about migrating from Apple's Mail app (Using .emlx format mail files) to Thunderbird (or any other non-Mail app). I looked at a few options and have decided, as strange as it may seem, that installing an IMAP service was the best solution.

read more

Has anyone ever run a service discovery and orchestration scenario like the one on page 7 of this document in any setting other than a demo? How far away from being able to do this in “real life” are we?

My overall impression is that it’s slower than the beta and slower than Safari 3. But that’s gut feel, not with hard data. The “loading” bar is also far less obvious now, and I always find myself wondering whether Safari has stopped working because it takes a while to even start showing activity.

Clear just closed down.

Here’s how it worked while it was in business. You paid $200 for a one-year membership. You underwent a big, complicated background check to prove that you were extra-super-trustworthy.

In exchange, in a few big airports, you got to skip to the front of the TSA line for screening.

Now, you didn’t skip the screening itself. You still went through the X-ray machine and had to remove your shoes, belt, pocket contents, laptops, and plastic quart ziplock bag of toiletries.

You just got to cut to the front of the line.

A few people signed up. In certain airports, it was, indeed, worth actual money to cut to the front of the line.

This wasn’t Clear’s actual business plan. The actual business plan was that Clear would do detailed background checks on travellers, who would then be trusted to bypass security completely because they were extra-super-trustworthy.

Now, the TSA doesn’t even trust pilots, who go through the same screening as the rest of us to make sure they’re not bringing something extraordinarily dangerous onto a plane like a 3.5 oz bottle of shampoo. Because, of course, with a little bottle of shampoo, they could make a bomb, which they could use to fly the plane they are piloting into a building, something that is impossible for mere pilots sitting at the controls of the jet.

So as it turns out, the TSA never actually agreed to go along with this skipping-the-screening thing, and ultimately, all Clear was allowed to do was get you to the front of the line.

At this point, and here’s the interesting part, at this point, a rational businessperson would say, “Well, does the Clear idea still make sense if we can’t actually let you skip the screening?”

OK, maybe it still makes sense to charge to skip to the front of the line. Maybe there’s a business model in that.

In that case, though, why did they still do background checks? It doesn’t make any sense.

The environment changed. It turns out that Clear’s business model of prescreening wasn’t going to be possible. But they kept doing it anyway. What kind of organizational dysfunction does it take to completely ignore the changed circumstances and keep at a money-losing business?

What’s even funnier is that Clear could probably have been profitable if they had just skipped the one unnecessarily stupid part of their business model: the detailed background checks on all their customers.

Nobody at Clear did any thinking. They had a business model, the business model wasn’t actually possible, everybody knew it, and they still plugged away at it. Thoughtless optimism. I don’t know whether to salute ‘em or laugh.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

I’ve been using Peaksware’s WKO+, a cycling and running training tool to manage data from heart rate monitors, GPS units, power meters, etc. Its a powerful tool with a clunky UI but I’ve gotten used to it.

You pay $100 for a “personal” license. Not a big deal to me since they basically have a monopoly on this software niche. I first installed it on my work computer to test the data from my daily bike commute. Cool it works. Then I went to install it at home since that’s where I’ll be using it. Works ok. I proceed to gather all my fitness data into their proprietary binary format.

Fast forward a few months. I’m reformatting the hard drive on the laptop and want to move all my data and software to my desktop. But installing WKO+ is giving me a headache (”Error: Too many installations”). The registration process takes a hardware fingerprint and your must active it via the web to get a registration code. However, hidden withing their EULA, is a term which dissallows the transfer of license to another computer other than the one to which it was originally installed. The second installation was just an allowance they make to allow for “hard drive crashes” and such.

Since neither of those machines would be available to me, certainly there would be a way to transfer it? After several progressively more desperate communications with Matt Allen at peaksware support, he informed me that there was no way they would transfer the license (the non-transfer clause IS in the EULA after all). I would need to purchase another license simply because I switched computers!

Here is my response:

Basically what you are telling me is that I can no longer use WKO+

without paying again. I get to use the software for a few months and

you revoke my right to use it because I buy a new computer! I am a

paying customer, trying to be totally legit here, willing to support

your business in exchange for a license to use your software and you

insist on screwing me over. Brilliant.

This is one of the most unprofessional and idiotic stances I have ever

seen from a software company. Your intention appears to be to screw

over your paying customers and milk as much cash from them as possible

- you might want to rethink that business model unless you want to

loose customers! I will never endorse, recommend or purchase another

product or service from peaksware nor will any of my family, friends,

teammates or readers once the word gets out about your disrespectful

policies.

There are numerous typical situations where a new copy of the software

would need to be installed including:

* Hard drive failure

* Operating system upgrades

* New computer purchases

* Extended traveling and touring (installing onto a laptop or netbook)

Now I fully understand why your policy is one license per computer. It

makes perfect sense. I have seen plenty of other software with a

similar licensing model. But they also allow to uninstall the software

and re-register it on another computer due to these circumstances.

There is simply no technological reason why you could not implement a

licensing structure that allowed the user more freedom to transfer

licenses while still preventing piracy. As it stands, your licensing

model treats paying customers like criminals if they happen to run

across any one of the above situations.

So, to sum it up - your foolish license policy has lost you one

customer and many future ones.

Good riddance.

So if you want to support a company that treats its paying customers like criminals because they get a new computer, go right ahead and support Peaksware. But anyone who expects to use software that they pay for even if they happen to buy a new computer should steer clear.

The real kicker is that all that work is locked away in their proprietary file format simply because of their draconian licensing. This is the real take home lesson to all software users (not just fitness geeks): If you lock your data away in a proprietary format and are beholden to a single company in order to access it, they can and will screw you. Always insist on open data formats, even if using proprietary software. Oh and always read the EULA carefully before clicking OK!

When I was setting up my linux system a few years ago, I did some research into filesystems and determined that the XFS file system, being particularly proficient in dealing with large files, would be ideal for my home directory. And it was. But the one factor I didn’t consider was portability. Turns out that there is basically no support for XFS in windows.

So how do you access your files from Windows if they are on an XFS partition? I had just shy of 1 TB of data to transfer so using my other linux box and transferring across the network would have taken forever. The solution I came up with is a bit convoluted but it has some real advantages:

1) Install Sun’s VirtualBox.

2) Download an iso for your favorite linux distribution (mine being Ubuntu 9.04)

3) Create a virtual machine from the linux iso

4) Install the VBOxGuestAdditions in the linux virtual machine.

5) Create a Share folder on the windows host and register it with the virtual machine. This will allow you to transfer files from the guest (linux) to the host(windows) You may have to manually mount the drive in the linux guest:

mount -t vboxsf share_name /mnt/share_name

6) Using the windows host cmd line, create a vmdk from the physical drive that your XFS partition resides on. In this case, PhysicalDrive1 corresponds to the second SATA connector. This will allow your guest OS to talk directly with the drive:

cd C:\Program Files\Sun\xVM VirtualBox
VBoxManage.exe internalcommands createrawvmdk
  -filename "C:\Documents and Settings\perry\.VirtualBox\HardDisks\Physical1.vmdk"
  -rawdisk \\.\PhysicalDrive1 -register

Once completed, you should see:

RAW host disk access VMDK file
C:\Documents and Settings\perry\.VirtualBox\HardDisks\Physical1.vmdk created successfully.

7) Make sure to add the physical drive to your list of hard drives in the linux guest options. Restart the linux guest virtual machine and your XFS partition should already be mounted. Now you can begin transfering files between your XFS partition and the shared folder on the windows host.

Whew. Lots of hassle for a simple file transfer, right! But the side benefit is that now you have a fully functional linux virtual machine with a shared folder set up to the windows host. Very useful - even when you must run windows, it helps to have a linux VM standing by!

Not sure why this didn’t occur to me before I wrote that last post but I tried the “pythonic” version of the code under the IronPython 2.6 Beta 1 release and it works!

lyr = Carto.LayerFileClass()
lyr.Open('C:\\test.lyr')
print lyr.Filename

Works perfectly now. So IronPython 2.6 promises to be a viable option for extending ArcGIS. My enthusiasm has been renewed.

Occasionally I find myself in the C#/.NET world in order to write code using ESRI ArcObjects. Today I was toying with the idea of automating the creation of ESRI Layer files (a file which defines the cartographic styling of a dataset). Of course they are in an undocumented binary file format, inaccessible to anything but ESRI software. So I pop open Visual Studio ….

I feel a nagging unease every time I type a set of curly braces. And VB just makes me insane. I prefer, of course, to use python. Luckily there is IronPython which runs on .NET - which means I could theoretically use it to interact with ArcGIS.

I only found a single working example of using ArcObjects through IronPython. But it looked promising enough to close Visual Studio and give it a go.

The first nagging problem is an IronPython-specific one. Relatively minor annoyance but you have to add the reference to a .NET assembly (library) before you can load it.

import clr
clr.AddReference('ESRI.ArcGIS.System')
clr.AddReference('ESRI.ArcGIS.Carto')
from ESRI.ArcGIS import esriSystem
from ESRI.ArcGIS import Carto

Now there is the issue of grabbing an ESRI license. A little verbose IMO but it could easily be encapsulated in a helper function to clean things up.

aoc = esriSystem.AoInitializeClass()
res = esriSystem.IAoInitialize.IsProductCodeAvailable(aoc,
         esriSystem.esriLicenseProductCode.esriLicenseProductCodeArcView)
if res == esriSystem.esriLicenseStatus.esriLicenseAvailable:
    esriSystem.IAoInitialize.Initialize(aoc,
      esriSystem.esriLicenseProductCode.esriLicenseProductCodeArcView)

Now that we’ve satisfied the demands of our proprietary license overlords, we can proceed with the real work .. in this case I just want to open an existing Layer file and see if the resulting object knows it’s own file path. Really simple, right?

lyr = Carto.LayerFileClass()
if "Open" in dir(lyr): print "The Layer object has an Open method but...."
lyr.Open('C:\\test.lyr')
print lyr.Filename

The Layer object has an Open method but....
Traceback (most recent call last):
 File "“, line 1, in 
AttributeError: ‘GenericComObject’ object has no attribute ‘Open’

Hrm. Looks like we’ve run across bug 1506 which doesn’t allow access to the properties and methods of a given instance - instead your have to work through the functions provided by the implementation. Grr…

Carto.ILayerFile.Open(lyr, 'C:\\test.lyr')
print Carto.ILayerFile.Filename.GetValue(lyr)

That is unwieldy, ugly and unpythonic. What’s the point of object oriented programming if you can’t access the methods and properties of an object directly? Since all ArcObjects applications are based on extending COM interfaces, this would be a major pain in any non-trivial application. Basically, until these .NET-accessible COM objects can be treated in a pythonic way, I don’t see any compelling reason to pursue IronPython and ArcGIS integration. Looks like its back to C# for the moment … (/me take a deep sigh and opens Visual Studio) … unless of course anyone has some brilliant solution to share!!

Did I say I would publish my performance results? I did. Here they are.

 

OK, this one I have to share. Here's two queries, the first with a syntax error in the WKT (oops!) and the second one correct.



First, as processed by MySQL:

mysql> select count(*) from tiger_roads_texas 

  where mbrintersects(geom, 

    GeomFromText('LINESTRING(452284 -1651542, 452484 -1651342'));

+----------+

| count(*) |

+----------+

|        0 | 

+----------+

1 row in set (0.00 sec)



mysql> select count(*) from tiger_roads_texas 

  where mbrintersects(geom, 

    GeomFromText('LINESTRING(452284 -1651542, 452484 -1651342)'));

+----------+

| count(*) |

+----------+

|        1 | 

+----------+

1 row in set (0.06 sec)

Now as processed by PostGIS:

tiger=# select count(*) from tiger_roads_texas 

where geom && 

  GeomFromText('LINESTRING(452284 -1651542, 452484 -1651342',2163);

ERROR:  parse error - invalid geometry

HINT:  "...RING(452284 -1651542, 452484 -1651342" <-- parse error at position 43 within geometry

CONTEXT:  SQL function "geomfromtext" statement 1



tiger=# select count(*) from tiger_roads_texas 

  where geom && 

    GeomFromText('LINESTRING(452284 -1651542, 452484 -1651342)',2163);

 count 

-------

     1

(1 row)

Can you spot the difference? Snark! Another one for the list.

 

I am doing a little benchmarking as a learning experience with JMeter and I will publish the throughput numbers in a few days, after I run the full suite I have developed on the various combinations of concurrency and insert/select ratios.



Because MySQL has so few functions that actually do anything (see the note here) there's not a great deal to test beyond raw performance. The early throughput results seem to indicate it's comparable for simple CRUD on one table, but for anything non-trivial it falls down.



Here's a basic spatial join: pull 23 roads from a 3.4M row line table and spatially join to a 66K row tract polygons table, calculating the sum of the areas of tract polygons found. There are spatial indexes on both tables.

mysql> select sum(area(t.geom)) 

from tiger_roads_texas r, tiger_tracts t 

where 

  mbrintersects(r.geom, GeomFromText('LINESTRING(453084 -1650742,452384 -1650442)')) 

and 

  mbrintersects(r.geom,t.geom);



+-------------------+

| sum(area(t.geom)) |

+-------------------+

|  1260394420.00453 | 

+-------------------+

1 row in set (9.43 sec)

And in PostGIS:

tiger=# select sum(area(t.geom)) 

from tiger_roads_texas r, tiger_tracts t 

where r.geom && GeomFromText('LINESTRING(453084 -1650742,452384 -1650442)',2163) 

and r.geom && t.geom;



       sum        

------------------

 1260394420.00684

(1 row)



Time: 5.574 ms

Those are both "hot cache" results, after running them a couple times each.

 

A WMS performance benchmark has been a staple of FOSS4G conferences for some time. In 2005, it was IMS vs Mapserver. In 2007 it was MapServer vs Geoserver. And in 2008, a grudge MapServer vs Geoserver re-match.



For 2009, we hope to continue the MapServer vs Geoserver tradition, and are inviting other WMS servers to join the fray. We are hoping to have ArcGIS Server in the mix, perhaps MapGuide, perhaps DeeGree. The participants are assembling on a benchmarking listserv.







However, right now we are stuck trying to find a location for our gunfight – we need an OK Corral. Our preferred corral would have the following characteristics:

• One or more dual-core processors
• 4Gb of more of RAM
• Centos or RHEL
• Remote ssh access for participants
• Root access or sudo for participants
• Not virtualized

In addition, we will need a second server on the same network segment for generating load (would still need remote access, but would not need a beefy machine). Due to the nature of the participants (global) and the timelines (several months) we would need sole use of the corral until the testing is complete in September.



If you have a corral you can donate for the shoot-out, let me know!



Update: We have received a generous offer from the US Army Corps.

 

Another disastrous consequence of inaccurate spatial information… Not only can you accidentally tag your neighbor as a criminal, now it appears that sloppy spatial data has lead to the wrong house getting demolished.

I’ve asked it before but its worth repeating … with all the recent advances in spatial data publishing, where are the advances in metadata and data quality assurance? How do you know where the data comes from, what’s been done to it and by whom? What is the intended use of the data? For the vast majority of the data being shoved out onto the web, these bits of metadata are sorely lacking.

Of course this case is more a matter of one person’s sheer stupidity; I’m not sure any caveats in the metadata would have stopped the wrecking ball!

Not the most exciting topic, but I’ve found myself knee deep in metadata standards as they pertain to CSW in the last couple of weeks. I’ve made some recommendations in the past for OWS metadata, which have helped in established publishing requirements for cataloguing. Starting to look at ISO metadata (data, service) makes you quickly realize the [...]

Dave Winer (in 2007): “Sometimes developers choose a niche that’s either directly in the path of the vendor, or even worse, on the roadmap of the vendor. In those cases, they don’t really deserve our sympathy.”

iSmashPhone: 15 Apps Rendered Obsolete By The New iPhone 3GS

When independent software developers create utilities, add-ons, or applications that fill a hole in their platform vendor’s offering, they like to think that they’re doing the vendor a huge favor. Oh, look, the iPhone doesn’t have cut and paste,  they say. Business opportunity! They might imagine that this business will be around forever. Some of them even like to daydream about the platform vendor buying them up. Payday!

The trouble is that only a tiny percentage of iPhone users are going to pay for that little cut and paste application. With any kind of add-on, selling to 1% of the platform is a huge success.

For Apple, that’s a problem. That means that the cut and paste problem isn’t solved for 99% of their customers. They will solve it, if it’s really a problem. And you’ll be out of business.

Filling little gaps in another company’s product lineup is snatching nickels from the path of an oncoming steam roller.

A good platform always has opportunities for applications that aren’t just gap-fillers. These are the kind of application that the vendor is unlikely ever to consider a core feature, usually because it’s vertical — it’s not something everyone is going to want. There is exactly zero chance that Apple is ever going to add a feature to the iPhone for dentists. Zero.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

From my latest Inc. column: “Giant corporations such as Google and Microsoft are like cities full of relatively anonymous people: You don't actually expect to see anyone you know as you walk around. Going to lunch on either campus is like going to the cafeteria at a huge university. The other 2,000 students seem nice, but you don't know most of them well enough to sit with them. Meanwhile, a typical lunchtime at my company is like Thanksgiving dinner: There's a big meal you get to share with a bunch of people you know and like.”

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

Andrew emailed to ask why we don’t have a StackOverflow DevDays day in New York City. That’s a fair question! There’s a big software development community here.

There are two reasons New York is low on my list. The first is cost. Hotels, venues, and catering are prohibitively expensive in New York. At a medium-class hotel, say, the Marriott on the East Side, giving everybody one coffee break with coffee, tea, soft drinks, and nothing to eat costs $23 per person [PDF]. It’s simply impossible to do a $99 one-day event in New York.

The other reason is attendance. I don’t know why, but techies in New York just don’t turn out for events at the same level as other cities. When we did the FogBugz world tour we had three times as many attendees in London as New York. Maybe New Yorkers are extra busy. But turnout is always surprisingly thin in the city.

It’s a particularly bad place to do tech conferences, too. Hotels are $600-$700 a night. Everything about putting on a conference is remarkably expensive. And half of your attendees wander off to visit the Statue of Liberty when they should be in your Python tutorial meeting.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

Whoa. Less than a month ago, we announced first Stack Overflow DevDays and opened registration to 300 people in each of five cities.

Well, that sold out pretty quickly. Ryan Carson, who is taking care of all the conference logistics, was pretty sure we’d be able to book larger event spaces, so we allowed even more people to register... we’ve got 2388 people booked worldwide so far, including over 800 in London, but it’s clear that with just five events we weren’t going to be able to accomodate all the people who want to spend a day meeting online Stack Overflow friends in real life and learning a little something about some hot new programming topics.

So, we decided to take the show to five more cities. Here’s the new schedule—click on a city to register:

Oct 07 Boston NEW!

Oct 14 Austin NEW!

Oct 16 Los Angeles NEW!

Oct 19 San Francisco

Oct 21 Seattle

Oct 23 Toronto

Oct 26 Washington DC

Oct 28 London

Oct 30 Cambridge, UK NEW!

Nov 02 Amsterdam NEW!

Tickets are $99 in the US, €84.75 in Amsterdam and £85.00 in the UK. A limited number of very cheap student tickets are available. If you already booked but want to change cities, email devdays@stackoverflow.com.

In the meantime, I’ve been working hard lining up speakers. I’ll be speaking myself in all ten cities, as this will be the launch event for FogBugz 7 and a new product now under development by the Fog Creek summer interns, who just arrived and are already coding away earnestly. Jeff Atwood will be speaking in San Francisco. Scott Hanselman will be speaking in San Francisco and Seattle. Jon Skeet will be speaking in London. The basic agenda should, more or less, include: Android, Python, Google App Engine, iPhone development, ASP.NET MVC, jQuery, FogBugz, Mercurial and DVCS, and a different speaker from academia in each city. There will be lunch, two breaks, and we’ll organize some kind of birds-of-a-feather post-conference meetups.

Anyway, at the current rate, it’s pretty clear these conferences will sell out long before the events themselves, so don’t wait until the last minute to decide.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

I'm a nice guy, I often raise ESRI's web services (formerly ArcWeb Services, now ArcGIS Online) when talking to clients about options for things like map services, geocodes and routes. It's my way of rooting for the scrappy underdog, the old paleogeographic home team, going up against the Google and Microsoft Bing behemoths.







But someone, please, tap the Redlands team with the clue stick... check out the fabulous new "free" services ESRI is offering to lure developers to their ecosystem!



Free geocoding! Yes! Free! And as many as 1000 geocodes per year. You read that right, kids, per year. Also routing! 5000 per year!



Compare with Yahoo!'s (aside, something about putting an apostrophe after an exclamation mark feels wrong) free API, which offers 5000 geocodes per day (Google offers 15000).



There's a punch-line in here somewhere, but I'm not sure where.



Update: Ray from ESRI notes in the comments that "... the limit of 1,000 geocodes is for geocodes done in BATCH MODE (ie: a request involving more than one address at a time). Place-finding, single address geocoding and single address reverse geocoding are not limited." I may have had it completely backwards, ESRI is not being too stingy, they are being too generous. I'm pretty sure there's lots of people who can script their computers into running lots of sequential individual geocoding requests ... in a "batch", as it were.



Update 2: Ray from ESRI further clarifies the meaning of "batch": "Batch geocoding really means that you are storing the results of your request locally, so you can use them again." So the "batchness" of your request is not governed by the size of the request, but by what you do with the request. (Wait, I've heard that somewhere before...) Comparing to the Yahoo! terms of use we find a similar restriction, which means the ESRI offering is the-same-only-better (fewer restrictions on non-"batch" requests). Better put away the clue-stick, nothing to see here, move along, move along.

 

I’ve been spending alot of time lately doing a CSW client library in python, which was committed today to OWSLib.  CSW requests can be tricky to construct correctly, so this contribution attempts to provide an easy enough entry point to querying OGC Catalogues. At this point, you can query your favourite CSW server with: >>> from owslib [...]

The Joel on Software Job Board has been working well since we launched it almost three years ago. It logs about 220,000 unique visitors every 21 days, including many passive job seekers who have RSS subscriptions.

But a few employers place ads and just don’t find anyone. I’m pretty sure we’re the only job listing service in the world with an unconditional money back guarantee, so these people call us and we give them their money back. It’s not a lot, usually just two or three a month, but I’d still prefer to have a wider audience for these job listings as long as it didn’t diminish the quality of resumes.

In the meantime, Stack Overflow has been attracting a huge community of smart developers—we’re running over 3.7 million unique visitors a month.

So today we’re launching a new Stack Overflow job board and a Server Fault job board, which will both be linked up with the Joel on Software job board. Any ad placed on one appears on all three sites. This will be a great way to recruit great programmers and a great place to get great jobs.

It’s not much of a secret, but Stack Overflow is already a great place to find good programmers, because you can see how good people really are by reading the answers that they post. I’ve noticed a lot of people putting their Stack Overflow reputations on their resumes, and we’re starting to hear stories of people who got jobs through the site. Jeff and I are committed to building features to make this easier in the next “six to eight” weeks. For example, I’ve always hated traditional resumes, which just don’t give the right kind of information about a candidate. If you wanted to hire an iPhone developer, would you rather know that person’s Stack Overflow stats in the iPhone tag and read their answers to technical questions? Or would you rather know where they went to college?

If we pull this off, getting jobs in the tech industry will be a lot saner.

In the meantime, if you are hiring, do yourself a favor and try placing an ad on Stack Overflow. (FAQ) If you’re looking for a job, check out the listings.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

Bill Dollins nails it:

In general, the perception of risk equals the presence of risk.

Like "truthiness", risk is in the eye of the beholder. Which leaves me working to help sell a software-based solution for a problem that exists, for the most part, in people's heads. Open source software vendor, or psychiatrist, you be the judge. Also, potential hole in the business plan – what happens when people finally wise up?

 

Server Fault is now in public beta!

When Jeff Atwood and I launched Stack Overflow last fall, we really wanted it to be a site for and by programmers. But the engine behind the site, the Q&A engine with voting, editing, and tagging, could obviously be used in a lot of other professions.

The first field we picked is close to our heart: system and network administration; as programmers, we often end up doing system administration ourselves. And it’s the perfect domain for a Q&A engine... there are a million detailed problems that depend highly on lore to get right. There’s no way to accidentally discover aspnet_regiis.exe -I until someone shows you the trick. How much time have you wasted trying to figure out which process is holding a file open preventing you from deleting an otherwise empty directory? Can you use dd to clone a disk drive?

Thus, Server Fault. If you already have a Stack Overflow account, you’re all set up, although your reputation score, badges, and favorite tags are separate. It has all the great features from Stack Overflow which I talked about at Google last month (video).

Jeff: “I am sorry to inform you that you may be a system administrator or IT professional.”

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

I never thought I would feel sorry for the Beast from Redmond, but they seen to be regularly plumbing new lows. Microsoft Bob, you have a new rival for Worst Marketing Brainwave Ever, and his name is Bing.



(And now I have to change all my presentation slides to refer to "Bing Maps for the Enterprise"? Flibbitty Jibbets for the Enterprise? Google at least managed to turn their silly name into a verb before plastering it everywhere in sight. Bing?!?)

 

Anselm just posted what appears to be a random thought on twitter:

Are you more generative than consumptive in your particular field? … Create more than you consume?

In open source, I often rephrase this question as “Are you a source, or a sink?”

There are many people in the community who contribute more than they consume. Organizations, individuals, etc. There are also many sinks in the community — since entropy is every increasing, this seems a forgone conclusion — and one of the key things that causes an open source project to succeed or fail is the number of sources or sinks.

I personally try very hard to be a source in all that I do, rather than a sink. One way that I do this is that I try very hard to always followup any question I ask — for example, on a mailing list, on an IRC channel, or what have you — with at least two answers of my own. This means that, for example, when I hopped into #django to ask about best practices for packaging apps, I stuck around, and helped out two more people — one who was asking a question about PIL installation, and one about setting up foreign keys to different models.

Now, in the end, my answers were simple — no one with even a basic knowledge of Django would have had problems answering them. But by sticking around and answering them, I was able to make up to some extent for the time/energy that I consumed from someone more familiar with the project, by saving them from needing to answer as well.

It is often the case that users trying to get help will claim that once they get help, they will ‘contribute back’ to the community by, for example, writing documentation. This never happens. Though there are exceptions to every rule, it is almost always the case that users who ask a question, prefacing it with “I will document this for other users”, never follow through on the latter half. The exceptions to this — or rather, the alternate cases — are cases where a user has already invested significant research, and likely already started writing documentation. Unless the process is started before the problem is solved, it is almost universally true — in my experience — that the user will act as a sink, taking the information from the source and disappearing with it.

I work very hard on supporting a number of open source projects that I work on. Though my involvement lately has been more hands off — by doing things like writing documentation instead of answering questions, acting as a release manager instead of fixing bugs, and so on — I work very hard to keep the karmic balance of my work on the positive side. I believe that this pays off in the long run — I have somewhat of a reputation of being helpful, which is beneficial to me since it means I’m more likely to receive help when I need it. I also work to keep karmic balance high on the part of the organization I work for, since many of the other people in the organization are less able to keep karmic balance high.

These rules don’t apply solely to open source — I have the same karmic balance issues going on in my work inside of MetaCarta — but I maintain the same attitude there. Coming in with the idea that it is okay to be a sink can lead to a nasty precedent. In the end, I think that everyone loses. Sinks — both in open source and other karmic ventures — will eventually use up the karma they start with, and be left out to dry. It is the case for more than one person that they have extended their information seeking without contributing back beyond the point where I am willing to continue to support their information entropy.

I joke sometimes about giving out “crschmidt karma points”. Though I don’t have an actual system in this regard, I do quite clearly delineate between constant sinks, and regular sources, and grey areas in-between. I try to stay on the source side, and I encourage anyone else to do the same — even if it’s only by answering easy questions on the mailing list, or doing a bit more research on your own. Expecting other people to fix your problems, in open source or otherwise, is simply a false economy of help, since in the end, it simply doesn’t work.

In an otherwise unrelated conversation, Howard Butler produced this short quote on effective software project names:

A name need not describe what something does (MapServer, I stare in your general direction :), but it needs to be distinctive enough to discriminate ...

This reminded me of the ArcGIS Server demo at Where 2.0, the Jack-and-Jeremy show (by the way, IRC consensus was that Jeremy Rocks, too bad he was off-line (and on-stage) and couldn't bask in the love) where a number of examples of searching for services were started from Google queries of the form: "mapserver something".



So, a search for "mapserver idaho" picks up this ArcGIS service pretty high up, a search for "mapserver floods" turns up this ArcGIS flood service, and so on.



This seems like bad news for the Google Karma of Mapserver Classic, and also bad news from a customer conversation point-of-view. "We're standing up [a] Mapserver" now has a couple of pretty distinct meanings.

 

This was my first Where 2.0, so unfortunately I cannot provide any longitudinal data. Apparently the early expectation was for a conference in the 1000 attendee range, and this gathering seemed closer to 500 – the economic downturn is biting hipster location freaks just as much as the rest of the economy.



A lot of the idle chatter after sessions seemed to focus on the notion that there weren't any "game changing" ideas being presented, which is probably more of a commentary on the rate of change in the industry over the past five years than on the quality of the presentations. My feeling is that the game changed only once, with the roll-out of Google Maps and (more importantly) the associated API. Everything since then has been watching the interaction of ripples and waves on the pond arising from the initial event. Long story short, we were a demanding and jaded audience.



I very much enjoyed watching the talks while monitoring (and silently heckling) on the #where2.0 IRC channel. The real-time commentary could instantly contextualize even generic industry boiler plate. One of the things the IRC channel noticed early on was that there were some repeating themes, which could be used to form a "where 2.0 drinking game":

• "twitter", take a sip
• "iPhone", take a sip
• "Google", take a sip
• "monetize", take a drink
• "find a coffee shop", take a drink
• "in the cloud", take a drink

The repeated references to "iPhone" form one of my big conference takeaway thoughts: ubiquitous geo-location technology (broad markets for consumer devices that always know where they are) is no longer a "future development", it's the present. We knew it was coming, for a long time, but now it's actually here.



Some of the applications of ubiquitous geolocation are really, really dull. Every framework and operating system is now adding a "geolocation API" which does little more than spit back a location-you-are-in-right-now. So Google demonstrated a "blue dot" on their maps, that takes you do your current location. Microsoft demonstrated Windows 7 location API, that provides all apps access to your current location (subject to various privacy controls). Nokia showed similar stuff. I met a Mozilla developer who is adding a location API to Firefox, same story.



Another predictable (but not dull) application of ubiquitous geolocation is spatial data mining. We will see more and more data mining going forward, but at Where I had my first exposure to the results of this technology, in a pair of talks on Wednesday.



Jeff Holden started with his talk on "Footstreams". Jeff did a great job of providing a terminology and explanation of the power of analyzing and clustering individual spatial tracks. His conceptual parallel of "foot streams" to "click streams", the temporal tracks of information we leave behind while browsing the web, is very powerful. Unfortunately, his service has access to a relatively small amount of voluntary data, so the analyses weren't very convincing to me.







Greg Skibiski, from Sense Networks provided the data meat for Jeff's conceptual sandwich. Sense Networks has access to heaping piles of data that telephony providers get from their subscribers (involuntarily). The telcos provide anonymized data to Sense Networks, who analyze it to provide a segmentation of the market based on spatial patterns. Greg's talk left me with a massive case of data envy – my inner statistician wants a terabyte of location data for Christmas.







Finally, Ted Morgan from Skyhook dropped an interesting nugget in the midst of his talk about consumer geolocation use patterns. He showed a graph of the number of location apps calling Skyhook over time. Basically, the use of geolocation has hopped onto a steep geometric growth curve over the last 12 months, since the introduction of the App Store. Location apps are now mainstream and the long-awaited LBS tsunami has begun. More devices (Android and Android store, Nokia and their store) are just going to further accelerate this, but's not the future anymore, it's the now.



A number of the talks on the morning of the second day were around "location based social networking" applications. The age of the people on stage, and the kinds of applications they are developing brought up two thoughts on the IRC back-channel that I thought were fun.



My observation was that it is odd that introverted geeks are laboring long hours to create applications that make it easier to "meet people". What kind of self-respecting introvert wants to go around meeting people? (Incidentally, the Skyhook talk also included a hilarious example "use case" of an iPhone app that let you both call for a cab and specify your destination, so (as Ted noted) you could get from point A to B in a cab without ever having to speak to a person. Sold!)



The follow-up observation from IRC participant "rabble" (I never linked that nick to a real person, unfortunately) was that the location based social networking start-ups are all solving a problem (keeping in touch with, and meeting up with, a diverse network of friends in a big urban area) that is of greatest important to a fairly narrow cultural niche. Why are we devoting so much venture capital to solving the social problems of young white collar professionals who have recently moved to a new city?



Final assessment, I went to Where looking for exposure to a side of geospatial that I don't get at other location conferences and that was fulfilled. The Silicon Valley culture is definitely more open source aware, thinking about non-traditional uses of location (in usually non-surprising ways), and maintains a sense of self-regard which is almost as high as my own. Altogether an enjoyable week, many congratulations to conference chair Brady Forrest in putting together an entertaining program.

 

... but they rarely order a second cheeseburger.



My dinner tonight in the warm pre-Where 2.0 air of San Jose:



Martini, $8.50

Beer, $6.50

Beer2, $6.50

Cheeseburger, $2.95



Chance to swap IT bullship with blogebrity James Fee, priceless.

 

Via Geoff Zeiss, those granola sucking hippies in the City of Vancouver council have decided to to promote open data, open standards, and ... open source?!?







I always get a little worried when politicians start actively mucking around in the operational details of government, but to be fair it is hard to put a hard black and white line between where policy stops and implementation begins. Is mandating open source a policy, or is it telling city workers how to do their jobs? Council could also pass a resolution not allowing the "q" key to be pressed on Fridays.



In this great debate, I think my position is this: mandating open data (basically an enhanced form of Freedom of Information) and open standards (without which Freedom of Information is moot) make sense as policy matters. Mandating open source crosses the line, dealing more with "means" than with "ends". City managers will eventually figure out on their own that the value proposition of considering open source is high. Council might be (hell, they are) right on this issue, but defining what tools are to be used to achieve operational goals is a slippery slope, and there's lots of places council will be wrong in the future on if they go down this path.



For a great example of why you don't want legislators telling the executive precisely what to do, see congressional earmarks.

 

Stack Overflow has been going nuts—after just six months in business, we’ve had 3.5 million unique visitors per month. We’ve starting thinking about how to get that great tribe of developers together in the real world.

We decided to launch a series of Stack Overflow events: the first gathering of the tribe of great developers making Stack Overflow so successful that over 90% of questions get answered [video].

It’s going to be in October, in five separate cities. In each city, we’re planning a one-day event.

We decided to cram as many diverse topics as possible into a single day event. Like a tasting menu at a great restaurant, we’ll line up six great speakers in each city.

This is not going to be just a Java conference or a .NET conference or a Ruby conference. This will be completely ecumenical. We’ll have somebody to introduce Microsoft’s new web framework, ASP.NET MVC, but we’ll also get someone to talk about writing code for Google’s new mobile operating system, Android. And in each city, we’ll find one local computer science professor or graduate student to tell us about something new and interesting in academia.

For smart programmers who are interested in learning about something a little bit outside of their own immediate field, this is the conference for you. We’re doing it in the spirit of Byte Magazine. Remember Byte? Every issue covered a wide range of topics and technologies. Sadly, Byte disappeared, to be replaced by Mac-only magazines, IBM-PC only magazines, even Microsoft SQL Server-only magazines.

The conference is for programmers. The conversation is going to be hard core. Speakers are going to be writing code.

Putting on these conferences is really expensive—conference centers can charge you $1000 for one urn of coffee. That’s why typical developer conferences can cost $1500, plus travel, hotel, and $73 for the Internet access in your hotel room. With the current worldwide recession, that just isn’t gonna fly. Many great conferences, like SD West, have been cancelled. Attendance is way down at the conferences that survive.

So I got together with Ryan Carson, whose company, Carsonified, has been putting on great conferences like FOWA, and we tried to figure out how little we could possibly charge for this thing. The answer: $99 per attendee.

That’s it. These one day conferences are just $99. You can bring your whole team for less than the cost of a normal conference.

We’re doing it in five cities, so you may not even have to travel. We’ve got room for just 300 developers in each city:

October 19 San Francisco

October 21 Seattle

October 23 Toronto

October 26 Washington, DC

October 28 London

Register for Stack Overflow DevDays right now. Given the huge demand and the limited number of tickets, I expect all five cities will sell out pretty much instantly.

OK, here’s the FAQ:

Who will be speaking?

I’ll be speaking in every city. Jeff Atwood will appear in San Francisco only. We will line up about six speakers in every city.

What will be the topics?

The topics haven’t been nailed down yet and depend on speaker availability. I’ll do my best to get speakers on as many of the following topics as possible:

• Android
• Objective C and iPhone development
• Google App Engine
• Python
• jQuery
• ASP.NET MVC
• FogBugz 7.0
• Mercurial and Distributed Version Control

Each talk will be fairly introductory but will be intended for advanced developers. If you already know about a topic, say, iPhone development, you can wander outside and hang out with the other iPhone developers in the hallways.

Is this the same in every city?

Not exactly. Some topics will be repeated—my talk will be the same in all five cities—but we may not have the exact same topics and speakers in each city. We aim to have at least one local speaker in each city.

I’m hungry!

Me too. I’m always hungry. Breakfast, lunch, and afternoon tea are included in the price of the ticket. Half of the fun of this conference will be meeting other Stack Overflow members.

Where are the events?

We haven’t booked halls yet. The exact location will be announced. There is a small chance we may have to adjust a date a little bit if we have trouble booking a space.

Note: If your company is in one of these cities, and has a presentation space that is suitable for 300 people, we’d love to have you as a sponsor. Please email devdays@stackoverflow.com.

What should I wear?

Make a T-shirt with your StackOverflow identity and reputation (see sample at right). You can order them here or make your own.

Are there academic discounts?

There are a very limited number of subsidized student tickets at $10. Due to the cost of putting on the event, I regret that there are only a limited number of student tickets in each city.

How can I help?

Ah! Thank you for asking. We need lots of help to pull this all together.

• We need volunteers in each city. If you live in one of these five cities, and would be able to help check tickets and just generally be useful during the conference, please email devdays@stackoverflow.com.
• We need sponsors for the meals and breaks. If your company would like to be a sponsor, please email devdays@stackoverflow.com. This is a great way to recruit superstar programmers: sponsors will have the opportunity to set up recruiting booths at the conference.

I have more questions.

That figures. Email them to (you guessed it) devdays@stackoverflow.com and Natasha at Carsonified will get back to you.

 

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

Magnetometer + GPS + accelerometer == device that knows exactly where it is and where it is pointed. So, augmented reality, here we come.

 

If terrorists can use Google Earth to bring down Western civilization, I'm sure pirates can find something useful to do with this.



Update: Title Reference

 

I recently (well, two months ago) and wrote down a great deal of thoughts about the architecture for web mapping we are pushing at OpenGeo, and it's up on the website now.



http://opengeo.org/publications/opengeo-architecture/

 

Update: I think the magnitude of the evil can only be appreciated if you see the JSP (yep, that's all of it, that's my "middleware"):

<%@ taglib uri="http://java.sun.com/jsp/jstl/sql" prefix="sql" %>

<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<%@ page contentType="text/x-json" %>



<sql:query var="rs" dataSource="jdbc/postgisdb">

${param.sql}

</sql:query>



{"type":"FeatureCollection",

 "features":[

<c:forEach var="row" items="${rs.rows}">

 {"type":"Feature",

  "geometry":<c:out value="${row.st_asgeojson}" escapeXml="false" />,

  "properties":{

  <c:forEach var="column" items="${row}">

   <c:if test="${column.key != 'st_asgeojson'}">

    "<c:out value="${column.key}" escapeXml="false" />":

    "<c:out value="${column.value}" escapeXml="false" />",

   </c:if>

  </c:forEach>

}},

</c:forEach>

]}

Update 2: Yes, I am being a bit sarcastic. Being able to compress the layer between the Javascript and the database into something this narrow is diabolical, and only possible because there is so much smarts in OpenLayers. I, for one, welcome our new hipster Javascript overlords.



Update 3: The "evil" is passing SQL unmediated from your browser directly into your database. It's fun in a workshop (which is what I wrote this abomination for) but it's not to be let out of the lab, lest global pandemic ensue.

 

“Even as competitors like Circuit City go bust, B&H remains packed with loyal customers. And that makes me very happy. For a business owner, there's nothing more satisfying than watching honest dealers expand their operations while the schmucks, with their going-out-of-business markups, go down the drain.”

From my Inc. column: Why Circuit City Failed, and Why B&H Thrives

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

The best conference of the year just got better! They've chosen me as a keynote speaker! (My turnoffs include soggy popcorn, rainy days and false modesty.) See you in Sydney, mates! Throw another shrimp on the barbie! That's not a keynote address, this is a keynote address.



Postscript: I just realized I learned everything I know about Australia before 1985.

 

OSGeo (your Open Source Geospatial Foundation) will be on the exhibition floor at Where 2.0 this year. I volunteered to be the "event manager" for OSGeo at Where 2.0 this year, but all the heavy lifting was done by Alex Mandel preparing for the American Geographical Society conference earlier this year. So thanks to Alex's hard work, we'll have some posters and brochures and handouts for the exhibit hall.



Most importantly, we'll have ourselves. Not booth bunnies, per se, but we have beautiful minds. Arnulf Christl, the OSGeo President, is coming in from Germany to attend, so there will be lots of good conversation to be had. Come ask us any question about open source, and we'll be sure to give you an answer!



I'm also giving a workshop on spatial databases and web mapping. In fact, I'm preparing the material right now. Oh, how I hate preparing material – the being done is so much better than the doing.

 

Here’s a video of a talk I gave at Google last week about StackOverflow.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

What have I been doing for the past month? Not much PostGIS, as the growing list of 1.4 milestone issues attests. Mostly residing in the fifth circle of marketing – web content.

 

Recently, the OpenStreetMap project put out a very successful call for donations to upgrade their physical database infrastructure, from a dual-core Athlon with 8Gb of RAM and lots (~1Tb) of disk, to a quad-core Xeon with 32Gb of RAM and heaps (4Tb) of (15K RPM) disk.



The speedy success of the hardware appeal (target reached in less than three days) was pretty impressive, but what really perked my (PostgreSQL fanboi) ears up was the news that the new hardware was going to run PostgreSQL, instead of the MySQL database OSM has used from the start. As of April 19, OSM is running their new API live on PostgreSQL.



So, why has OSM abandoned the worlds most popular open source database? I asked the OSM folks, and this is what Tom Hughes of OSM told me:

Personally I've been very frustrated with MySQL from when I first got involved with running things. Some of the problem was of our own making in that we had a mix of MyISAM and InnoDB tables (originally everything was in MyISAM) and some tables were using MyISAM features that meant they couldn't be easily moved to InnoDB.



On top of that it seemed that virtually any non-trivial query would completely defeat MySQL's optimiser.

The comment about a mix of tables really hits home, since so many MySQL features are split across table types. Want transactions? InnoDB! Want full text search? MyISAM! Want spatial? MyISAM! Want spatial or full-text and transactions? Tough. The devil is in the details. When asked: does MySQL support spatial, transactions, full-text? the MySQL answer is "yes", "yes", "yes", but the reality in production is not nearly so clear-cut.



Note that OSM is not using PostGIS for the main database at this time (their current data model of nodes and ways wouldn't get much leverage from it) but it is used for other processes like OSM tile generation. And a growing number of people on the PostGIS users list seem to be using osm2pgsql to extract data from the OSM production server for rendering / analysis in PostGIS.



So, welcome OSM, to the PostgreSQL community!

 

Announced yesterday, this release closes 92 bugs, and adds some new goodies. Next stop: MapServer 6.0 Written from home: window.onload=function() { var point322 = new GLatLng(43.663194, -79.575653); var map322 = new GMap2(document.getElementById("map322")); [...]

Someone, please call the FTC! I may post my blog on Google, store my mail and calendar and notes on Google, teach people how to make maps using Google, but even I have my limits.



The Google Profile leverages Google search monopoly (you listening, FTC?!?!) into a you-must-join social network. I like LinkedIn as a professional social network, but this is going to blow all that out of the water. Want to be found? To control your professional image? Yes, having a decent personal site will still be important, but now the starting point for a personal search (Google Search) is also the end point (Google Profile).



Can you afford to not have Google Profile? I don't think I can, and I think that stinks.



Update: Apparently I wasn't clear about my point: because Google controls search, their Profile is a must-have for anyone who expects to be looked up on the internet (independent contractors, mobile professionals, me). It is not optional. It's a Google world. Comply.

 

Happy Earth Day! If you missed the earth sandwich three years ago, here's a chance to re-live the dream!



The Original Challenge

The Find-the-Opposite Tool

The Winning Entry Announced

The Winning Entry



Mmmmmmm. Earth sandwich.

 

Somehow, I missed this video in the geoblog meme-stream, so for all the rest of you who also missed it:







And since I'm posting YouTube links, you must watch this unlikely Scottish talent show contestant blow the audience and judges away (figuratively).

 

http://www.sun.com/third-party/global/oracle/.  I wonder what this will mean for MySQL? Written from home: window.onload=function() { var point314 = new GLatLng(43.663194, -79.575653); var map314 = new GMap2(document.getElementById("map314")); map314.addControl(new GSmallMapControl()); [...]

Sitemaps are a valuable way to index your content for web crawlers.  GeoNetwork is a great tool for metadata management and a portal environment for discovery.  I wanted to push out all metadata resources out as a sitemap so that content can be found by web crawlers.  Python to the rescue: #!/usr/bin/python import MySQLdb # connect to db db=MySQLdb.connection(host='127.0.0.1', [...]

I use the logged_in_or_basicauth snippet for a lot of my work, and had had some problems with it since I started using mod_wsgi in place of mod_python. Thanks to this post, I now know why my basic auth under mod_wsgi isn’t working: lack of WSGIPassAuthorization On in my Apache config.

Thanks to the author of that post! Also, thanks to Google, since without it, I’d never have found it.

For a number of years now, I have watched with longing as Where 2.0 slipped past me on the calendar. I love the idea of a geo-conference where most of the attendees are not in the government/land management/asset management realm. Unfortunately, trivial matters like the births of children and so on have gotten in the way of my attending, thus far. But not this year! This time out, I'm going to Where, and I'm going to be teaching a short course on spatial databases. Spatial SQL is a powerful tool, so powerful that you don't require much more than a spatial database and a web mapping API to do some nifty stuff. OSGeo will have a booth this year, so if you're at Where and you want to talk databases, track me down there.



Reminder: If you're reading this before Wednesday, you still have time to register before the early bird rates go away.

 

Alot of tools out there have some sort of configuration which, at run time, is read and used in the process accordingly.  When writing tools, my config file format has always been something like: title: My Tool # commented out line description: This is my tool.  # another comment Since I’m using Python for much of my scripting these [...]

On my last trip through SeaTac, I thought my thin copy of WiReD was a physical harbinger of the recession. This time, the state of my flight from Victoria was a pretty clear sign.







Oh boy.

 

Leaving the house this evening, my daughter presented me with a daffodil from the garden, and I put it into my lapel. Little did I know that this would cause me to be sent through the "special line" at customs: the daffodil is an agricultural product.



On the up-side the officers were uniformly good humored and enjoyable and basically waved me through (who can be cross with a man with a daffodil from his daughter?) so it was actually a positive in an otherwise normal drab transit of SeaTac.

 

Remember when "coverage" was the holy grail? "Boy, if I could just read/write coverage, I could interoperate with the GIS folks no problem."



And then things seemed to get better. Most data was in "shape files" and there was a published specification which more-or-less matched the data generated by the ESRI tools. And lots of folks were using ArcView, so you could share "avl" files (layer style) and "apr" files (project file) and even write third-party tools to consume them directly (they were just a funny text format after all).



And now they are back-sliding. Layers and projects are "lyr" and "mxd" binary formats generated by ArcGIS and not consumable by third party tools. Data is starting to be moved into "file-based geodatabase" (FGDB) and the only tools that can peer into those are Arc*. (The old geodatabase had a lot of drawbacks, but at least you could open it in a third-party tool.) There are no specifications for the data formats, for the layer formats, for the project formats.



I thought we in the computer profession had this discussion and came to a consensus: vendors that tried to use proprietary file formats as a lock-in were to be avoided if possible and castigated if necessary. Even old Microsoft had to play the "open format" game, at least in mixed company.



It's not just data, though the rise of the closed FGDB format over the last couple years is bad enough. The "metadata", the style files and the project files, are a key piece of "operational data" that can be incorporated into workflows automatically – if you can read and write them. With format lock-in in place, the only way to automate your work flows is through the vendor-approved channel.



Really, I thought it was decided – they don't get to do this to us anymore.

 

Open source has some real problems in the market, and most of them seem to reside between the ears of the people making corporate decisions on software. Really good article on procurement perceptions here from a Plone support company (what, Plone needs support? :)

The bit that Simon said that made the bulb go on in my head? He said that most companies set out on a procurement process to procure a software license. That is what software is to them: licenses. That is what they know, and that is how they think. If you've ever had the fun of dealing with the contracts/procurement department of a large company you will recognise this thinking. And so with that goal in mind, of course they end up succeeding in their quest to procure a license for software

—Matt Hamilton

Why do we get so uptight about rings that touch themselves? After all, masturbation is perfectly normal for a young ring. Healthy even.

 

Three years ago, I was invited to New Zealand to speak at Webstock, the local web conference. Now, let me tell you honestly, I go to a lot of these conferences, and the speakers are often saying the same things, or I’ve heard them before, so it’s hard for me to sit still when the other speakers are talking.

But once in a while, someone gets up and just blows me away. I’m absolutely riveted and I start scribbling notes furiously. One or two good speakers like that, and you’ve got a conference that’s worth going to. In Wellington, New Zealand, it was Kathy Sierra. Kathy co-created those Head First programming books with all the crazy pictures and diagrams and the cover stock photography of cool-looking models taken from a very high angle. Kathy used to write a blog called Creating Passionate Users which, though dormant, is still awesome. At Webstock, she got up and started talking about brains and lions and all kinds of things and it was the most amazing speech I ever heard.

So I’m incredibly excited to announce that Kathy will be speaking at this year’s Business of Software conference (Nov 9-11, San Francisco).

And it’s not just Kathy. Neil and I share a philosophy that instead of doing a conference with one or two keynote speakers and a lot of filler, we want to have a conference where every speaker is worth going out of your way to hear. With any one of these speakers, it’d be worth going to this conference. With all of them, it’s a once-in-a-lifetime opportunity.

We’ve got Geoffrey Moore. His seminal book Crossing the Chasm discovered the problem growing high tech companies making the leap from the early adopters to the mainstream. You won’t find many successful startup founders that haven’t read and learned from Moore.

We’ve got Don Norman, author of The Design of Everyday Things, one of the most influential books about usability in the history of the field.

We’ve got Paul Graham, who created one of the first internet ecommerce sites, who invented a spam filtering algorithm that has kept our inboxes relatively clean, and who has inspired and funded a whole generation of entrepreneurs through his seed investment fund Y Combinator.

We’ve got Heidi Roizen, who co-founded T/Maker, a PC software company that she grew to over 100 employees, and is now CEO of SkinnySongs.

We’ve got Jennifer Aaker, a professor of marketing at Stanford, Michael Lopp, a.k.a. Rands in Repose, who wrote the book on managing humans at high tech companies.

I’m just getting warmed up. You’ll also hear from Ryan Carson, Paul Kenny, Dharmesh Shah, and the mysterious entity known only as The Cranky Product Manager.

I’ll be speaking, too.

I’ve never seen anything like this ... a one-track conference with two and a half days of awesome speakers, every one of whom would be worth going to San Francisco to hear. This one will sell out. The conference is $1995, but the next 65 people to register save $500, so register sooner rather than later.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

You know Spring has really returned to Victoria when the cherry blossoms come out. I can't begin to say how happy I am not to live in a place where turkey vultures are the harbingers of Spring.

PowerPoint is a way to make gibberish look important.

– my 12 year old daughter, Alicia

We’re always looking for good programmers at Fog Creek, but right now we could really, really use a top notch Win32 (C/C++) developer to join the Copilot team. I know you’re out there. Apply now! Thanks!

 

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.