Index of all posts.

Post Archive

Page 3 from June 28, 2013 to May 23, 2014.

Where Does the `sys.path` Start?

Constructing Python's `import` path

Importing modules in Python seems simple enough on the surface: import mymodule looks across the sys.path until it finds your module. But where does the sys.path itself come from?

Sure, there is a $PYTHONPATH variable which "augments the default search path for module files", but what is the default search path, how is it "augmented", how does easy_install or pip fit into this, and where does my package manager install modules?

Read more... (2 minutes remaining to read.)

Posted . Categories: .

Dictionary Building for Word-Search

Mining Wikipedia for a "geek" lexicon.

One of the local pubs styles itself after geek/nerd culture (e.g. sci-fi, fantasy, and board games). The back of the coasters feature a word-search. It reports to contain 45 "geek words and phrases":

Writing code to solve a word-search isn't particularly tricky (if you remember to use a prefix trie) as long as you have a list of words to find, but in this case we are given no such clues.

But, since we are tremendously lazy, how can we solve this with code anyways?

Read more... (2 minutes remaining to read.)

Posted . Categories: .

Autocompleting Python Modules

Simplifying the search for modules to execute from a shell.

The last few times I overhauled an execution environment I required people to execute the bulk of their tools via python -m package.module instead of python package/module.py (to enable the development environment).

The downside is that you lose shell autocompletion, which can be a big deal if you have dozens of tools that you only occasionally use.

This addition to your ~/.bashrc fixes that.

Read more... (1 minute remaining to read.)

Posted . Categories: .

@classproperty

Is this an amazing, or terrible idea?

Lets define a classproperty in Python such that it works as a property on both a class, and an instance:

class classproperty(object):

    def __init__(self, func):
        self.func = func

    def __get__(self, obj, cls):
        return self.func(cls, obj)

It can be used thusly:

class Example(object):

    @classproperty
    def prop(cls, obj):
        return obj or cls


x = Example()
assert x.prop is x
assert Example.prop is Example

Is this a good idea, or a bad idea?

(Hint: I don't know.)

Posted . Categories: .

Linear RAW Conversions

The first step when using photos in your computer graphics.

Real world photography is a fantastic and easy source of data in computer graphics and visual effects, be it for textures, backgrounds, or light maps. There is one complication that is very easy to overlook, and tricky to get right: linearity.

In real world light transport (and the simulated version of it in our various renderers) the math of light operates in a linear manner. That is, light source A with intensity 1 and light source B with intensity 1 will combine to intensity 2 (in whatever units those numbers are).

However, image capture and display devices do not work in a linear space. This is mostly historical, and maintained for backward compatibility for the content that has been produced in the past, but we must still deal with it to this day. In order to be immediately useful for 99% of cases, non-RAW image formats (as produced by your camera or the RAW converter) have the inverse curve already baked into the image so that they appear similar to the real world when viewed on a display.

It is this curve that must not be applied in order for our photography to be representative of the world (as far as the math is concerned).

I won't get into what "gamma" means, or scene-referred vs. display-referred imagery, so for an in-depth look at linear workflows see Sony Pictures Imageworks' 2012 SIGGRAPH course nodes (PDF) and this FXGuide article.

So how do we get linear output from our DSLR?

Read more... (3 minutes remaining to read.)

Posted . Categories: .

How Not to Store Passwords: Adobe Edition

A quick peek at the Adobe password breach.

Recently, someone broke into Adobe's network, stealing source code and part of their password database. It later turned out to affect at least 38 million people.

I always try to investigate to see if I am personally impacted by these leaks. Usually, that means submitting a carefully hashed password to some online service built to inform you if you were part of the leak. This time, however, the entire file was readily, and easily availible (by which I mean my Twitter feed included a link to it, several times).

So... why not take a peek?

$ grep mikeboers cred
78626857-|--|-adobe@mikeboers.com-|-+8pqoIav87bioxG6CatHBw==-|-Same scheme.|--

While not surprising, I am still dissapointed to be included.

Digging just a little bit deeper, there are some very troubling things in this file.

Read more... (2 minutes remaining to read.)

Posted . Categories: .

Yet Another Reason I Hate Time

While the Y2K bug turned out to be a bust, the concept behind it is pretty solid.

I have always had a lingering concern for 3:14:07 am on January 19th, 2038 UTC, as that is the last second representable in signed Unix time, and time in systems that still use that representation will wrap around to January 1, 1970. (See the year 2038 problem.)

At the time, 70 years in the future seemed infinitely far away.

There is no chance this will still be running in 70 years.
— Likely said by someone on the POSIX committee.

Unfortunately, that type of thinking just cost a very expensive space probe. 8 years into the Deep Impact mission that was designed for a single event, its clock rolled over, the probe couldn't figure out where the sun was and so couldn't point its solar panels at it, and lost power.

"Basically, it was a Y2K problem, where some software didn't roll over the calendar date correctly," said A'Hearn. The spacecraft's fault-protection software (ironically enough) would have misread any date after August 11, 2013, he said, triggering an endless series of computer reboots aboard Deep Impact.
"NASA Declares End to Deep Impact Comet Mission", National Geographic

As always, anthropomorphized space probes pull at the heart strings:

Posted . Categories: .

Git Metadata In-Band

Moving configuration and hooks into the tree.

When writing tools for git, or while working on more complex projects, I often need to work with the git configuration or hooks.

Unfortunately (or so I sometimes feel), the configuration and hooks are not stored in the tree (the version controlled part of repo), and must be installed out of band. This easily leads to a non-uniform configuration across clones.

Let's get that information in band!

Read more... (1 minute remaining to read.)

Posted . Categories: .

Render Heatmaps

Are pictures worth 1000 statistics?

Debugging rendering issues can be particularly problematic. Many times, the efficiency of standard debugging procedures (e.g. printing intermediate values, or using a debugger) fall apart at the sheer volume of data they will produce when you are calling them millions of time per frame.

Often, intermediate values can be dumped out via an AOV (i.e. to another image), and inspected as an image. For example, if you were interested in how long various parts of the image are taking to render vs. the others, you could create a heatmap such as:

In this particular example, however, there are a few drawbacks:

  1. RSL does not have any timing functions;
  2. every shader would need to be modified in order to collect these stats; and
  3. you generally only receive information from the front-most surface.

I set out to resolve those issues.

Read more... (4 minutes remaining to read.)

Posted . Categories: .

The UX of Coupon Codes

How to (try to) avoid confusion and apprehension.

Lately, I have been entering a number of coupon codes on the web, and have found them to be infuriatingly lacking in one respect: the character set used for the codes is not easy to type.

I'm sure we have all questioned at one point if a character in the code was supposed to be a 0 (zero) or O (upper-case letter O), or a 1 (one), I (upper-case letter I), or l (lower-case letter L). Usually you just pick one, and usually you get it wrong the first time.

I find it particularly strange, that Starbucks would go so far as to recognize this problem, but not actually fix it:

Starbucks Error Dialog

As a developer, there are a few increasingly dramatic ways to deal with this.

Read more... (1 minute remaining to read.)

Posted . Categories: .

View posts before June 28, 2013