Determining if a Manager is being used by a related object.

Recently, I wanted to figure out if a custom manager was being called by a related class or the original class. As an example, consider this abbreviated code:

class MembershipManager(models.Manager):
    def get_by_type(self, type, user=None):
        # figure out if we need user or not...

        # then call self.get_queryset().get()
        pass


class Membership(models.Model):
    user = models.ForeignKey('auth.User', related_name='memberships')
    m_type = models.CharField(max_length=200)


    objects = MembershipManager()

So... how does get_by_type know if it's being called as "user.memberships.get_by_type(foo)" or "Membership.objects.get_by_type(foo, user=bar)?" Well, I knew that related managers pre-filter on the object they're related to, so I just had to find the code that does that.

It's right here in the ForeignRelatedObjectsDescriptor class. The interesting bit is:

    def related_manager_cls(self):
        # Dynamically create a class that subclasses the related model's default
        # manager.
        superclass = self.related.model._default_manager.__class__
        rel_field = self.related.field
        rel_model = self.related.model

        class RelatedManager(superclass):
            def __init__(self, instance):
                super(RelatedManager, self).__init__()
                self.instance = instance
                self.core_filters= {'%s__exact' % rel_field.name: instance}
                self.model = rel_model

This is a class factory that, in this case, subclasses MembershipManager when creating the attribute user.memberships (actually, that's not quite true. user.memberships is actually the ForeignRelatedObjectsDescriptor which hands out this dynamically created, and unnamed, subclass of MembershipManager. The descriptor is used as a proxy so that User.memberships doesn't return anything that actually is a related manager because it would be misleading at best.)

So, long story short, if self.instance exists when `get_by_type` is called, we're in a related object subclass and you can check if self.instance is a User object, in which case you can then allow the caller to skip the user keyword argument.

This is probably documented somewhere on the Django site, but I couldn't find it easily.

Google+ Sign In: "The mere fact that you call it that tells me you're not ready."

This blog post is a rant, of sorts.  How I plan on making this worth the read is to offer a few small suggestions, some positive feedback.

Starting about a week ago, "History" is now "Google Sign In", so it seemed as good a time as ever to make this post. Even though you could always sign in with Google if a site supported their OpenID+OAuth API, now when you sign in, "Moments" or app activities can be pushed to your Google+ account. I'd like to point out that I hope I'm wrong about how clunky some of this stuff is. Feel free to correct in comments if I've gotten something wrong.

So the basics of Google Sign In are pretty straightforward and are probably familiar to everyone reading this. Much like Facebook Connect and Twitter Anywhere, you can allow a 3rd party site (maybe Rdio.com or Vimeo.com) to get personal information about you so that you can skip setting up a username/password on that site, and possibly allow that site to send information (theoretically on your behalf) to Facebook, Twitter, or now, Google Plus.

So if you go to a website that allows you to sign in with the Google Sign In, you'll see something like this, initially:

Pretty basic stuff, looks much like Facebook connect; asks you to give some personal info and additional permissions[1]. I'm assuming it runs on basically the same guts of the OAuth stuff Google's had for a long time. The part that's interesting, is the part where it allows you to make "app activity and reviews available via Google" visible to ... fill in the blank. Your Circles[2], of your own choosing, can then view your activity! Wonderful!

Where can they view this wonderful activity? Well, that's where stuff starts to get dodgy. You can, if you somehow know to do this, travel to the user's profile, click off their stream onto their "About," then scroll ALL the way down and click on an app, listed there.

That is, if you decide 'hey this user might have interesting thing in this app' ... because how would you ever know that? There's no preview. There's no indication of activity. Connected empty apps look just like very busy apps. Hell, there's no real way that a new user would even consider looking there for their own stuff.

After 3 screens of scrolling, we get this brilliantly inviting link to click on for activity.

Yay, now I can see a very uninspiring summary of my activity in one app. 

Needless to say this is pretty crap; it's difficult to find and does not draw you in at all. So now it's time to talk Facebook. How does Facebook solve this? Facebook has the same kind of events or moments or activities[3] or whatever you want to call them. Two properties about Facebook activities:

1. They go in the activity feed on the sidebar.
2. They show up on your Timeline (profile) in the month that they occurred inline with all other Facebook Timeline items such as shares, likes, etc.

The Springpad app on my profile with activities listed. It's visually interesting, but photo apps look even better.

So, should Google Plus do this as well? Google Plus' profile is no where as refined or space efficient as Facebook's Timeline, so it could definitely use some love.

Great use of Space!

So I say "yes." But, if you want to differentiate, Google Plus, maybe make these boxes configurable! Name them "Activity Windows" and make it so you can group activity from different apps in them, name them something, etc. You know? Then you could have something like this:

Part of the API should probably allow 3rd party apps to at least initially install an "Activity Window" too, because new users shouldn't really have to configure these things on their page, configuration is confusing.

Now, part of Google Plus' touted appeal of App Activity is that "Sharing is selective; spraying is just spam." This addresses part two of Facebook's activity component, that activities are broadcast to people and it's spammy.

However, Facebook has solved that. A long while back, they started putting things that weren't necessarily full shares into the sidebar. That is, if someone likes a photo, comments on a post or listens to something on Spotify, that activity goes in the sidebar and I can turn off the sidebar! That is, literally, problem solved on that front.

I realize maybe they don't want to invent a separate stream, as Facebook did, for just activities. I don't blame them, sometimes I want things to show up in my profile, but not on other people's streams.

All of social networking is advertising, it's just that the percentages vary. Some things are 90% self promotion and 10% informational. Sometimes things are 70% informational and only 30% self-promotion[4]. That being said, I might want to note on my profile "here are some of the things that I read, if you're interested and my friend" but I don't necessarily want my, say, Libertarian friend who worships Ron Paul to see in the activity stream that I just read a primer on Modern Money Theory, to make up a mild example.

Just the idea that someone could take offense to something that shows up in their stream, from me, without me explicitly saying "show this article saying technology luddites are ruining the world to my friend Tom who hates technology" is enough to explain why I ended up un-linking my Springpad app from Facebook. So I like explicit sharing that goes into streams, but there's no reason that display and sharing have to be mutually exclusive.

Which brings us back to what happens if you choose the second option when signing into a site with the new Google Sign In. What happens when you choose the "only You" option for who sees app activity. Well, then, the app still sends activity/moments to your account, but to share that you need to go back and sift through them by digging into your app activity in the same awkward way listed above to find my activity, and then manually share it. I won't even know if I have new activity, as far as I can tell! 

I mean, you can definitely make the argument that having the app write activity that you can then share is not necessarily the most useful thing in the world in most cases, but at least make it visible for me to share it easily when it does happen. So this is the other major failing I would correct. My suggestion, at least the only one I can think of right now, is add more notification icons to Google. That's one of the best parts of G+ is that I can see notifications in my normal day-to-day workflow (Gmail, Google Docs, etc.) without explicitly checking. Maybe something like this?

In addition, ultimately, I think 3rd parties you've connected to should be able to send separate notifications and harness the power of notifications unified via Google. Google encourages people to use the moments API to push things like comments, so why shouldn't those sites be able to notify you about things surrounding those activities like someone replying to your comment or liking it? So I would actually propose three notification boxes: Google Plus Notifications, Moments/Activity Notifications, 3rd Party Notifications.

I can imagine there will be people saying I've got this all wrong, I just want them to copy Facebook, etc. There's no end of kool-aid drinkers out there about Plus. I just read someone saying effectively "Google is awesome because it makes it's products hard to use!"[5] But I want Google Plus to work. I really do, I've been trying to use it as my primary social media outlet, both for my own sanity and because I trust Google more than Facebook. It simply hasn't worked for me. I think I'll make a separate blog post about why that is and why Twitter/Facebook are still my standards, but I'm tired.

And look, G+ is, kind of inherently, copying what Facebook does. It has a stream, the sign in looks/works very similar to Facebook Connect, on and on. If you're going to go through the effort of adding something, don't half ass it or make it impractical/confusing in the name of being different or to not copy.

You're already letting apps write activity, it shows on the profile, etc. It's similar already. Call a spade a spade and just say "look, this is a good idea and social network sites that are looking to fill this niche have to have this." And then, you know, tweak it a bit. Keep the whole "activities aren't automatically shared with people's streams" thing, that's a great modification.

Tagging +Jonathan Langdale because, wisely or unwisely, he wanted me to notify him when I finished this.

1. And hopefully doesn't then prompt you to enter MORE information AND choose a password after you've ALREADY connected to Google Plus (hello tunein.com! Learn to internets!)
2. That's basically "Friends" in Google+ parlance.
3. Even Google can't keep the terminology on something they JUST released straight. The docs can't make up their mind whether it's Moments or App Activities:

4. Even if you think you're sharing an informative article, part of the impulse for doing it and part of the reward of people liking/interacting with the post is to say "look what an informed person I am, I think I'll share this to show you."
5. http://www.googleplusdaily.com/2013/03/how-sign-in-with-google-is-different.html#.UTWVuDA3tqA

Springpad.com: Pull browsing and streams of consciousness into notepads

So an idea came to me a while back for a site to build and then it turns out someone already built it. Which is great, I don't have to do any of the work! +Springpad is a web/mobile app that makes it easy to collect little bits around the web, text, pictures on your phone or on the web, and put them in notebooks which can be public or private, have comments, have multiple collaborators, etc.

I'll explain why I use this and why it's so neat, in my opinion, but first here's the details:

The web page is here.
The Android app is here.
The iPhone app is here.
The Chrome extension is here.

Why I Wanted It

So first of all, bookmarking kind of sucks. There's generally just a title and it's awkward and clunky to navigate large bookmark folders. Here is, seriously, some of my bookmarks in Chrome:

So that's no good, though it's easy and fast. But so many times I'm browsing around and I find an article that's interesting and maybe I want to reference it later, so where do I put it?

Well, I was using GetPocket.com for a bit, but it turns out that maybe shoving a bunch of stuff into one gigantic pile, which is what it does by default, isn't so hot and I ended up with a HUGE backlog of un-categorized stuff.

What of this was reference? What of it was for reading later? What of it was just documenting?

Pocket's simplicity is awesome if all you're doing is marking things to read and then deleting them as you read them, but that wasn't really my use case. (NB: Totally possible I was just misusing Pocket, but the interface didn't guide me to do things "better" other than simply adding tags which seems a bit less than stellar.)

Enter Springpad

So with Springpad you make "Notebooks." They get organized into a personal library:

There's a bunch of options for how the pads display once you click into them, but mine (all web links currently) mostly look like this right now:

This app/site solves the following problems for my problem domain:

• The Android app can push to Springpad from anything that can "share." Photos, tweets, web pages, etc. can be easily shared. (I haven't tried video or audio yet.)
• The Chrome plugin makes it easy to add pages as you browse.
• People can now see what I've been reading in a much more visually interesting way than Google Reader and organized how I want it to be.
• I can add notes, TODO lists, etc. to these pads. (Remember, they can be public or private.)

You CAN share directly the things you Springpad, but I like the fact that you don't have to share them for people to get at them, that's key, to me. I don't have to post something you might disagree with or upset you on your stupid Facebook wall to be able to have a public place on the Internet which collects things I've read/found interesting for others to see!

You can make a journal, a collaborative wall with really interesting pictures, or save things for reference later. There's actually a ton more things you can do, but I just think the idea is very cool.

Disclaimer: I don't actually care to engage in heated discussion about things I put in my pads. It's not for you, it's for me. If you think it's neat, feel free to say that, but I'm sharing it on Springpad because if I like an article that you disagree with YOU WERE THE ONE THAT WANDERED INTO MY PAD IN THE FIRST PLACE so I don't care to waste brain cycles hearing why my opinion or the article (which I may or may not agree with) is "so horribly wrong."

Post RICON 2012 idea fragment: embed CRDTs in data

So RICON2012 is over and it was pretty fun. I thought the length of the session was just right (two days) and I thought that the amount of people was pretty good, too, especially for a first convention.


There was a lot of talk of distributed data structures (or "Conflict-free replicated data types" [CRDTs for short]), which is neat. I missed some of the talks, but I was thinking about this and it seems to me that CRDTs are nice, but you often don't want to keep track of just one data structure.

For example, you might store user interactions in Riak as a simple JSON dictionary. Clicks, adds to cart, etc.

Your key might be uid_category or uid_campaign, with a data set that looks like: {clicks: 8, adds: 9, ... }

This might make a lot of sense if you're doing simple analytics to figure out how popular a product or product category is or whatever. However, you might not want to break up these multiple statistics into many keys, because whatever process is figuring out how to rank things may want all the data but might not want to make, say 6 key fetches to get them (especially if you're doing batch processing over lots of users.)

In addition, you may have a lot of data coming in so obviously a CRDT counter implementation would be useful for each one and you would probably want siblings turned on. This makes me think you might want to embed the operations you're performing in the data (this might be meta-data, but I'm not sure.) That is, append to your data something like this:

{clicks.increment 1, adds: increment 1}

Then, if you have siblings, you assemble the corrected data from the CRDT log in the siblings. Easy! This is a limited application, for sure, and it's just something I thought of off the top of my head, but I'm wondering if tooling around this would be interesting/useful.

Django 1.3 model ID not showing up after .save() with PostgreSQL

So there was an interesting change with how Django selects the value for an ID with the PostgreSQL back end between 1.1 and 1.3 (or perhaps 1.2, I'm not sure.)

The function to get the ID of the newly saved model used to read:

def last_insert_id(self, cursor, table_name, pk_name):
   cursor.execute("SELECT CURRVAL('\"%s_%s_seq\"')" % (table_name, pk_name))
   return cursor.fetchone()[0]

But in 1.3 (at least), it reads:

def last_insert_id(self, cursor, table_name, pk_name):
   # Use pg_get_serial_sequence to get the underlying sequence name
   # from the table name and column name (available since PostgreSQL 8)
   cursor.execute("SELECT CURRVAL(pg_get_serial_sequence('%s','%s'))" % (
       self.quote_name(table_name), pk_name))
   return cursor.fetchone()[0]

The difference is that 1.3 asks PostgreSQL what the name of the sequence is based on the table+pk name. The name is a little misleading as the PostgreSQL documentation indicates. It really should be like "pg_get_owned_sequence" because that's what it's asking: what is the sequence owned by this column?

It turns out if you have a pk that was created as an INTEGER and then assigned a default value of the next value of a sequence... you still don't quite have the same thing as a pk defined as serial because the column won't own the sequence.

You can rectify the situation thus:

ALTER SEQUENCE sequencename OWNED BY table.pk

And that will fix it. Then the pg_get_serial_sequence call will not return NULL, and your model will get an ID.

This is all something you can figure out with the documentation on the web, but I figured I'd put it here to maybe help someone, somewhere.

The unruly "Health" industry: some examples.

The strange medical economy we have going in the US is highlighted in one case by the story of the drug Makena, which is used to help prevent premature births. Women particularly at risk of giving birth prematurely rely on this drug.

The original drug in question, hydroxyprogesterone caproate, was made for years as Delalutin by Squibb (which became Bristol-Myers Squibb).

However, in 1999, Delalutin was discontinued by Squibb (which had been making and marketing the drug since 1959.) It was OK because "compounding pharmacies" ... which is pharmacies that can make their own compound drugs, filled the gap and sold the drug for a fairly low cost ... something like $10 to $20 per dose.

Then, a company called KV Pharmaceuticals decided it was going to market the drug by getting it FDA approved. Once they did that they'd have the sole rights to make this drug in the USA, due to patent law, as I understand it.

... Then they increased the price from about $15 a dose to $1,500 per dose. That's about $30,000 for the entire regiment that women need if they're at risk of premature birth.

Fortunately, Senator Sherodd Brown, The March of Dimes, The American Academy of Pediatrics, the American College of Obstetricians and Gynecologists, and the Society for Maternal-Fetal all protested and the FDA agreed not to restrict compounding pharmacies from making and selling the drug.

Except for these people raising their voices, the cost increase would have just happened. It was clearly the intent behind KV Pharmaceutical to charge $30,000 for a drug regimen to prevent early birth which had previously cost hundreds. (They've since cut the price in half to a mere $15,000).

Read some more details about the story here.

Health journalist Julie Applebee wrote a piece which simliarly serves as an example of the ecosystem of the health care industry, here recounted by Dr. Gregg Bloche on NPR's Fresh Air. I'll let him tell it:

Back in 2007, a so-called 64-slice CT scanner came into use, really high-resolution. Cardiologists loved it. They started buying it for their offices, and...

DAVIES: What does that mean, 64-slice?

Dr. BLOCHE: It means 64 different levels of images very close together. A CT scanner works by taking a cross section of the part of the body that it's scanning.

DAVIES: So you get a three-dimensional picture, in other words, right?

Dr. BLOCHE: Exactly. It's a three-dimensional picture that's made up from a whole bunch of slices. Imagine looking at cross sections, at multiple levels of something. You can do a cross section of the brain at multiple levels. And the closer the cross sections are to each other, the finer the resolution on the CT scan.

So 64-slice is just a fancy way of saying a really high-resolution CT scanner, so high-end resolution that you could put people in it and look at their hearts and figure out how much coronary artery blockage they had without putting a catheter inside their arteries to pump dye into their arteries, which was a rather scary way of assessing levels of coronary artery blockage.

So the cardiologists loved this. They could buy this machine and charge a huge amount for it and show beautiful pictures, stunning pictures, colorful pictures, of people's coronary arteries and the degree of blockage.

Only thing is that this test, it turn out, only turned useful for a very small number of patients who had serious coronary vascular disease. Medicare agreed to pay for it only for this small number of patients.

But the cardiologists exercised their right to petition their government. They lobbied Congress. Seventy-nine congressman from both parties wrote a letter to the agency that runs Medicare, saying: Cover this thing. Medicare soon rescinded its limiting rule and agreed to cover the test much more broadly.

So politics plays a big role in the movement of expensive technologies that yield only tiny benefits right into the marketplace. And then the developers of these technologies know that. And so they keep spending. The investment bankers know it. The venture capitalists know it.

You can read the whole interview here.

Stories like this are very important to the national health care debate and the government finance debate. The Health Reform Act passed recently doesn't seem to address any of these concerns. I'd love to see evidence that it does, but if there is, I've missed it. Nor does it touch on the ecosystem of R&D, which I haven't got time to go into now, which seems to have an equally sordid past of cannibalizing publicly funded research.

Until the ecosystem of health care is fixed and until we go back to funding research done at universities around the country, it seems like we're headed for a downward spiral of increasing costs and no one is talking about it on the national stage (probably for some of the reasons Dr. Bloche pointed out.)

David Buss in conversation with Richard Dawkins

This is a really fun video series, David Buss has done excellent work for years but I've just recently started reading some of his work after having seen it referenced time and time again in other pop sci books about evolutionary biology/psychology.

One of the amusing things about this series is that while Dawkins had a huge influence on Buss, and Buss even uses The Selfish Gene in the classroom, Dawkins is clearly not up to date on the most recent evolutionary psychology research and theories. In a reversal of Dawkins talking with Singer, Buss is very diplomatic with his answers.

For reference, Buss' study of 37 different cultures is here.