After running through a few rather interesting interviews (one of which involved rewriting the Java collections API from scratch, one involved a shortest path graph traversal with some added edge-weights), I wound up with a new job the Monday after finals week at bConnected Software in Louisville, CO, where I’ve been working for the past four months or so.  I’ve had an absolute blast so far, doing a ton of Grails UI development, and I’ve learned a lot about user interface and experience design.  The company deals mostly with health insurance, which is proving to be a nicely complicated problem to solve, and I’ve gotten in at the right time, what with all the legislation surrounding the topic.  There has been quite the focus on Health Care Exchanges here, and we’ve been really working on defining the process of those in terms of both backend and UI/UX.

There have been a few little problems with the job so far.  The biggest by far is the commute – the drive averages about fifty minutes each way and is costing quite a bit, what with gas and tolls.  Since I was not a very smart kid over the last six or seven years, I’ve got quite a bit of credit card debt to pay down, so I wind up with relatively little money left after paying for the commute and paying down the card’s balance.  Other than that, corporate life is taking some getting used to: seventy hour weeks and business rules especially.

In other news, I did another experiment with rapid prototyping in Django and came up with http://characters.openfurry.org/ which is a good deal more complicated than my last experiment which proved to be pretty fun.  I wrote it after watching several furry artists deal with different ways of accepting information from commissioners regarding what they want drawn.  The result is a site which lets you manage a hierarchy of information about characters, from the characters themselves, to different morphs (basically a combination of species and gender), to potentially several descriptions of those morphs.  The site will also let you attach characters to different locations – places on the ‘net such as MUCKs and chat clients – and attach images to just about anything.  As an afterthought, I added a means for activity to be logged so that you could see what the owner has done recently with their characters/morphs/descriptions.

I had originally intended to use this site as a playground for Angular, a nifty new Javascript library that I’m quite taken with.  I ran into some snags, however, and did not get that implemented in my allotted time span, so it will have to wait, perhaps until this weekend.  In the mean time, I’ve been slowly poking through d3, the successor to Protovis, in order to provide some visualizations for the site, using this as a learning experience.  It’s proven tougher than I had thought, but definitely a lot more flexible because of it.  Another new thing I’ve been playing with is the Golden Grid System in order to lay the page out in a flexible manner without having to think about it too much.  Once I get some time, I’d like to get the Angular interface running, and maybe also play around with Twitter’s Bootstrap to make this a much prettier site than it is currently; though I’m partial to minimalist designs, as it stands now, I know a lot of people like flashier sites.

Finally, here are some thoughts from a commuter’s perspective:

• No one who drives a Saturn is happy to be on the road.
• Most people who drive Priuses and work trucks are pretty predictable drivers – I like that.
• Most people who drive Mazda 3s and 6s, Infinities, and Audis are pretty unpredictable – I don’t like that.
• “Arrest-me red” is a real color.
• Pickup drivers are usually somewhere on a scale from angry to smug SOBs; usually, the older, beat up pickups are smug SOBs and the brand new, super clean, very large pickups are angry SOBs.  This is not necessarily the rule, though.
• Audiobooks are awesome.  News radio is depressing.

Rather than be coy about my education, however, I’m combining the fact that much of my applying for jobs happens online with my resume into a visual resume that offers all the same information while show-casing my design and visualization abilities.

There’s a lot of conflicting information on how to structure a resume these days – should it be restricted to one page?  How personal should you be?  Avoid using ‘I’? Should you have an overview or skills section?  While I can’t say one way or the other what’s best for a personal resume (personally, I keep it to one page, have short overview and skills section, use ‘I’ while still being professional), I had a little more freedom working with visual resume.  Not only would it be divided up into separate pages  for clarity’s sake, I could take a little more time on each page to talk myself up and explain the accompanying graphics.

This project used, of course, Protovis, but I also included a map using OpenLayers and with CloudMade tiles.  It was nice to get back into the swing of mapping again, as I haven’t really touched that in a while with the library maps project being shelved.  And of course, it was fun to work with Protovis as always.  The biggest problem came up when I had finished the whole project, though, and started checking it in other browsers.  Chrome: great.  FireFox: great.  Safari on iOS: great.  Internet Explorer: …nothing.  Neither Protovis nor OpenLayers would work properly in IE8 64bit.  This could prove to be a problem.  The solution I’m working on is to export the graphics generated by Protovis (SVG format) to a format that IE does recognize and have an IE version of the page (done unobtrusively, of course, using IE’s browser-specific tags).  The plus side to this is that, since I will have the graphics already exported, I’ll be able to pull together a paper version of this resume that I can print out on glossy and use in some circumstances in person.  The down side being that I lose almost all the interaction that I have in place currently in the other browsers.  Ah well.

The smaller of the two projects was a very fast (about two hours) visualization of data regarding a disturbing punishment discrepancy between heterosexual/non-heterosexual teens by both the justice system and schools.  You can see in the vis that, in some cases, non-heterosexual teens can be two to three times as likely to be punished by schools or police.  Some reasons for this are debated in the original article that was pointed out to me.  The comments section on the article is rather distressing, I should warn.

The second, larger project has taken up most of my energy over the past few days.  A friend pointed me toward the results of the Furry Survey, which I hadn’t seen before.  The furry community differs from society at large in several very important demographic areas, from gender to sexual orientation (or at least openness regarding the same), to mean age.  With these important differences, I felt that a series of visualizations was almost necessary.  I’ve pulled those together into a dashboard that displays each visualization in miniature (not just thumbnails – the visualizations are reconstructed in miniature within the dashboard).  Check it out and find out some neat things about the furry fandom.  If you don’t know what furries are… well, Google carefully.

FurAffinity.net is a neat site full of neat artists and good community.  However, the statistics they provide for each user’s art are not only private, but rather lacking, being simply a list of numbers.  Sounds like a good job for visualization, though!  The numbers FA provides are views, favorites, and comments per submission.  Not only did I display those, but averaging them and normalizing for those averages gives a pretty good idea of relative popularity of each submission.  Users can view all four statistics in a steam chart, and also each alone in a bar chart.  I figured this was a good way to divide things up: trends are visible in general over time and one can explore specifics for each set.

There’s still some work I’d like to do, and I’m planning on collating the data I collect into a general graph of submissions on FA, but that’s for later.  It’s fall break and I’m still on the job!

You can see the result here.

I like it when data is already fairly organized, and I like it when it winds up being hierarchical.  I think we, as humans, are attuned to dealing with hierarchies as it is, so it makes sense when we can work with data that’s organized as such.  We’re also pretty good at spatial recognition, so it’s neat to play around with using area charts in a way that people wouldn’t normally expect.  This led to the natural conclusion of the tiered pie chart, or ‘Sunburst’ as it’s been called.  The fact that all of this is all so easy to work with in Protovis (their documentation aside, but no rants on that this time), certainly made this visualization easier!

I’ll be doing most of these challenges and posting their results here, along with any other little projects I can think up for learning this whole concept.  I’ve got a few ideas planned for when I’ve got the time!  They’ll all be archived at that new site, too, http://vis.mjs-svc.com

Many of those things can be considered ‘greater than the sum of the parts’.  When you look at politics, for instance, you can look at it as being different levels – federal, state, county, city, and so on – but you if you take all of those politics together, you get something that’s much more complicated than just the sum of the levels.  When thinking about legalizing marijuana, you can see that Colorado and several of its cities are all okay with the issue, and so medical marijuana dispensaries have popped up everywhere.  However, marijuana is still illegal at the federal level. Instead of it it just being two different laws, an interaction between the two laws occurs, with federal agents acting under the marijuana-is-illegal law and state agents acting under the marijuana-is-legal law.

The same goes for partisan politics: while states may have several republican elected officials, counties and the country may have more democratic elected officials.  The result is far more complicated than many people attempt to reduce it to.  Instead of this current election being a fight between socialism and free-market liberalism as the media is putting it, it’s much closer to being a fight between a loose amalgam of left-of-center ideas and a loose amalgam of right-of-center ideas.  Neither extreme – though they are represented – will win out due to the fact that the sum of the parts being much more complex than any of the parts would like to believe.  In other words, this is not good and evil (I’ll let you pick which is which), this is abstract thing and abstract thing.

Perhaps the reason that so many are so okay with talking so much about those things when they know so little is that they have a tendency towards behaving as fractals.  That is, you can start with the big picture and a general understanding of the concept – enough to form (and thus voice) an opinion – but you can drill down almost infinitely deep to find that the issue does not become clear cut, but instead becomes more and more complex with more and more magnification.  This is sort of the opposite of the previous point, leading to a weird sort of feedback loop: each of the parts is probably just as complex, if not more so, as the whole system.

A big theme in finance seems to be the discussion between those who are interested in macroeconomics and those who are interested in microeconomics.  With macroeconomics, the focus is on the flow of need and want through the entirety of the economy.  If you drill down, you get to microeconomics, which focuses on scarcity and how that affects decisions made by households, firms, and individuals.  Most stop there, but you can focus on scarcity itself and the concept of limited goods, or look at the concept of money as a representation of labor, or look at society as a collection of actors with needs and abilities and how they interact with each other, or even look at how each individual has a different internal representation of their worth to themselves, their family, their friends, their community, their country, and the world at large.  Needless to say, the entire concept of the financial system just gets more complicated the closer you look, and while self-similarities appear here and there, perhaps the actual complexity is just as infinite as the length of a fractal’s border.

Finally, it seems that it’s easier to form strong opinions about something the less you know about it, rather than the other way around. Perhaps due to seeing both sides of the issue, whether through research or exposure, it gets harder to be one-sided about an issue.  With the previous point in mind, one can look at the Mandelbrot set and generalize it as looking like a fat, spiky bird as seen from the top, but zoom in and change that generalized image as one goes, and of course this differs from person to person.

When I grew up, I was raised to think of religion as being a mildly malevolent group of forces determined to get people to act the same under the cover of benevolence.  Most of this concept came from my mom, who had had a difficult time with her family’s religion growing up.  However, the more I researched, the less I was willing to just believe this out of hand: it seemed that a good many religions focused on social justice, and even those where social justice wasn’t evident in the religion itself, schisms and portions within the religion had lead to groups that focused on social justice.

It’s easy to see how that would work from the other side as well, though.  Someone who was brought up in a religion that provided everything they needed for all of their life finally realizes that certain things about the scriptures or certain stories in the news don’t add up to a completely benevolent organization.  One of my good friends, John Wright, went through something like this with the LDS church: while much of the religion and many of the people provided the spiritual outlet that he needed in life, certain things began to irk him about the texts the church used and the way the institution was run.  Finally, after his first year of college at Brigham-Young University, he moved to Fort Collins to go to CSU and filed his apostasy officially with the Mormon church.

It’s difficult to say where this study leads us, since on the surface it appears to end in ambiguity about the things we know most about.  My composition professor, Dr. David, believes that it leads to the desire to study the issue more – specialization – and finally production or implementation of the ideas that remain, some of which are likely to be very impressive indeed.  Thus, a beginning composer might write a two minute piece in the style of his favorite composer, which he considers the best there is, but an experienced composer might write a twenty-minute large-ensemble work that incorporates new ideas formed in the process of his study of the subject.

Until the point when you know enough about something to form real ideas and not just blind opinions, though, the previous article stands: it’s a thing.

Anyhow, I’m an avid reader of FlowingData, because Nathan Yau, the man behind it, does some pretty awesome stuff.  His visualizations are clear and still aesthetically pleasing, and his concepts are always nice.  Of particular interest to me, when I first started reading, was your.flowingdata which is a means to track your own life through Twitter – for example, you can tell it when and how far you ride your bike every day and have it automatically generate a visualization of distances ridden over time.

Recently, however, he posted a little challenge of sorts.  Given a dataset, we, the readers, were to visualize it our own way and draw some conclusions from our visualizations (that, after all, being the point of visualizations).  I’d never done anything like that before for various reasons.  I didn’t want to learn a new domain-specific language such as R that would then require me to edit my results in the form of an image in some other program such as Gimp or Inkscape.  Also, Gimp and Inkscape have some quirks that I’m still learning, and I didn’t want to have to chose between learning those and buying Adobe CS.  However, I have been working quite a bit with Javascript recently, so it seemed to make sense that, when I found two libraries – Flot and Protovis – for visualization in JS that I go ahead and use one of these ‘Visualize This’ challenges to learn one of them.  It’ll definitely be helpful in the future.

The most recent challenge was visualizing data from the National Survey of Sexual Health and Behavior.  Given a small set of data – percentage of respondents in different age groups admitting to engaging in nine different behaviors over the past year – I worked hard to learn Protovis from scant documentation in order to pull together a visualization.  Since it takes place over three ‘slides’ and has text to go along with it, I’ll let it speak for itself here.

I think I did fairly well, given the fact that I wound up doing exactly what I didn’t really want to – learn a new DSL.  Granted, this one will be useful in my web design in the future!  With the time limitation of a due date and the fact that I was learning as I went, I didn’t quite pull off exactly what I wanted, and the trends I was interested in looking after weren’t as apparent I was hoping.  The problem was mostly due to inadequate documentation on Protovis – much of the documentation that wasn’t simply API documentation was either examples or brief write-ups about concepts in statistics as the applied to Protovis.  I learned most from the examples, after I learned some of the basics from the API docs.

I’ll probably find another dataset somewhere that interests me in order to visualize it soon, but I also expect that I’ll be implementing the visualization process in my own projects as well.  I’ve got lots of ideas.

Lets go ahead and combine this with the fact that it seems as though Planet Money, one of the financial podcasts that I follow, was created solely to explain the financial crisis to people in clear terms, seemingly a spin-off of This American Life after that show aired a few episodes regarding not only the financial issues, but issues of health care and housing.  We can also add in the additional reading and discussing that has been going on between a few friends and I about the problems involved in religion, and a few other friends and I about the problems involved in gender.

Finally, combine these with a few books I’ve been reading about disparate subjects but which all involve this concept of non-spatial, non-temporal ideas and we get the hole I’ve dug myself in now.

This thread, at least in the form it stands now, was started by a book I’ve read and re-read (and talked about ad nauseum at outings with friends), Anathem, by Neal Stephenson.  Stephenson is (in)famous for his habit of infodumping – that is, writing a story to espouse an idea, generally by wandering off topic (or having characters do the same) to the point he was trying to make about that topic.  Thankfully, he’s also a skilled writer, and so pulls this off well.  Again, thankfully, he doesn’t restrict himself to one idea per book, and so novels such as Anathem turn out to be wonderfully complex; even though they tend to be wordy about one subject for a few pages, they move onto another subject down the road without coming off as inconsistent.  The book aside, the pertinent idea that struck me was that of how we think of ideas that aren’t based in space or time.

Granted, one can claim that politics are based in space due to socio-political borders and in time due to the ever-changing political atmosphere, not to mention the fact that laws are enacted in time, but what does that really mean?  It seems to me as though the laws we follow are abstract concepts based on… what?  Morals?  Morals are even more vague than laws, because one can definitely say that a law applies to all citizens within a set of geographical borders, but that muddies things even further due to the fact that geographical borders are constructs of differences of opinion, armament, or physical features of the land.

Politics isn’t even just a set of laws that apply to citizens, though.  Politics boils down to a complex amalgam of ideas that strive to cross all aspects of life and (assuming that they’re well written/well intentioned/intended to be either/even written as anything but an afterthought) attempt to make it easier for disparate personalities and social groups to coexist.  Theoretically.  This is not even taking into account international relations, as politics have enough trouble dealing with interstate and interpersonal politics.  Policy is so large that it becomes worrisome that most people can boil their political stance down to an idea that can be stated in three or four words on a three inch by ten inch bumper sticker, or worse, a single person’s name – and not even that person’s name, someone else’s.

Finances are much the same way.  I recently finished Michael Lewis’ The Big Short, which was an excellent book and did well to explain the beginnings of the financial crisis in a character-driven fashion.  All well and good, but having finished the book, I’m left with this elegant, simple, and distressing picture of how sub-prime mortgages, collateralized debt obligations of the same, and credit default swaps betting against those CDOs work, I still can’t hold the entire picture of the crisis in my mind, never mind the entire picture of how finances work in so-called normal situations.  I’ve got a pretty good idea of how the stock market works, and I think that I understand how mutual funds work, and options… well, I can kind of understand the concept.  Bonds are beyond me.  I know that they can be divided up into tranches, tranches can be organized and sold based on what three ratings agencies say, and ratings agencies work with models which, from a programmer’s point of view sound woefully inadequate but are nonetheless likely more complicated than I could guess.

As for religion, I’ve been confronted on a daily basis for the past week with a street preacher on campus that pulls together an enormous and volatile crowd around him with an ease that only strikes me as disturbing.  The problem, as I initially saw it, was that he was preaching a subset of the Bible – specifically the subset that he was likely inculcated with (I realize that is a loaded word, but I’ll defend it in a second).  The crowd seems to be composed of three groups: those who agree with his ideas, those who oppose them, and those who are attracted to the conflict between the first two groups, listed in order from smallest to largest.  It is a college campus, after all.  When I listened to what he said – much ado about guys wearing pink shirts subverting gender roles and aligning themselves with the devil, a good portion about how physical scientists are distracted from the pursuit of Truth by their pursuit of science, how working toward a material, non-material but still non-spiritual, or just plain non-Christian way of life would certainly lead to hellfire – all I could think of was that quip about how many Christians are like those who scroll to the bottom of software license agreements just to click “I agree”.

Then I thought about how I had been raised.  Both  my mother and father were atheists (mom militantly so, dad more dismissively so), and so it’s no surprise that I think of preachers such as Brother Tom in terms of being “inculcated with an idea” or “dismissive of such large segments of the population” or just plain “nutjobs”.  I was raised by people who brought me up to be skeptical of the type of person that Brother Tom personifies.  I read the bible, find all the standard contradictions, find all of my favorite phrases so dismissed by these people (or worse from my point of view, rebutted with phrases from later on in the book taken out of context: Ecc. 9:10, which draws the reply of Rom. 12:11).

In short, with politics, finances, and religion – just to name a few – I find myself often in situations where I will never convince the other party of the validity of my position, but they will likely never convince me of the validity of theirs.  It seems to boil down to the fact that we’re dealing with these non-spatio-temporal ideas which, by virtue of their not being based in space or time, have no objective measurement.  In the book Anathem, the sconics assert that there’s no constructive way to think about non-spatio-temporal ideas: we can’t perceive them in any rational fashion, so we can’t think of them in any rational fashion.

Only we can, of course.  I was talking with a friend of mine tonight of search algorithms, of how it might be possible to think of latent semantic indexing (how Google and Shazam work) as a configuration space problem.  He seemed to agree only after much discussion and even then only with much in the way of clarification.  The root of the problem in this discussion is that I simply don’t understand the math used to describe LSI, but I do understand the concepts behind configuration spaces.  It is easier for me to hold in my mind the concept of a configuration space based on corpus of n documents than it is for me to work through the steps involved in some equation that I don’t even know how to type here.

As another example: the performance of music involves time and space, music itself is something that requires space and takes place over a period of time, but on the other hand, one can think of a score or a MIDI file as a vector representation of that rasterized idea that we call “music”.  MIDI deals with time in an abstract way, sure, and one can even play with panning, but one can also disregard both aspects and read the entire MIDI sequence at once to come up with an overall picture of the music, which we know as a score.  This is something I’m well acquainted with, considering how long I’ve been working with it.

In the end, this is the conclusion I come down to.  What I can do is think of something in the terms in which I understand it – it may only boil down to a bumper-sticker’s worth of information, but I can think about that much.  What I cannot, or rather should not do is use that understanding as the basis for talking.  I should not try to convince someone of my views on the financial world based only on a few podcasts and a book.  I definitely should not start an argument with Brother Tom based on my imperfect understanding of faith, much less the Bible.  Finally, I most certainly should not talk about advanced models of data with Joe while drunk!

What I mean to say is that we can only really speak about those concepts which we can hold entirely in our minds.  We so rarely do, and that’s part of being human, but I think it’s a wonderful goal to be able to speak only what you know.  Knowing, of course, may take the form of religious faith or of utter conviction without a substantial amount of proof, but speaking with knowing to back yourself up would make you strong indeed.

It confuses me that so many liberal, light, or lazy Christians talk so loudly about their faith.  It seems to me as firm agnostic that one couldn’t go a minute without thinking about, talking about, or living under the shadow of God if one were Christian.  It seems strange to me that someone deeply embedded in the financial world would have room in their mind to consider the socio-political effects of gay marriage, or the legalization of marijuana.  It surprises me that someone focused solely on politics, able to hold such a complex system in their mind, would be able to begin to judge music and call some good or bad.

“It is a thing”

That’s my response.

Someone can ask me about my views on politics and all I can even begin to say is that “it is a thing”.  If they want to argue a point which I understand – gay rights, music education, possibly sub-prime mortgage bonds – then I’ll be able to give them the attention that they want.  However, if they want to talk about markets and futures, social security and international affairs, the concept of the trinity as three individuals in one God, then all I will be able to say is “it is a thing”.  Yes, these concepts exist, and it is very likely possible that someone could hold their entirety in their mind, but that someone is definitely not me.

Gender stereotypes as unwittingly perpetrated by the targeted gender? Sure, I’ve got a vague idea of what I think, but it is a thing.  I can’t hold that complex a topic in mind without losing track of how many times I have to breathe per second without losing track of the concept of MIDI as a vector format for music.  How the New Testament relates to the Old – does it supersede or do they coexist, or does it depend on chapter and verse?  Well, alright, I suppose it depends on your view of how the Bible was intended to be read, but that really just makes it a thing.  I can’t comprehend the entirety of the book at once, so I have to deal with it in smaller units.  I may, may be able to hold Ecclesiastes in mind, but probably not Ecclesiastes and Romans, never mind the concept of Paul.

I know that most people don’t work that way.  I know that most people are able to hold enough of a topic in mind to form an opinion and state it loudly and firmly in the face of an opponent with utter conviction, but I don’t think that’s quite the best way to do it.  Of course it’s possible, and of course their’s a possibility that it will get you somewhere.  Maybe you’ll change the other person’s mind.  More than likely, though, your speaking without a more complete knowledge will hinder your cause and lead to more exasperation than anything.

What I am not saying is that one should embrace apathy about the subjects that one doesn’t know anything about, or that one should focus on such a tiny point that one loses the bigger picture.  It is a burden of responsibility in order to care about something because then you must learn everything that you can about it.  And after you learn everything about the subject and feel comfortable talking about it, you must be able to be corrected, shot down, or crushed by the arguments of those who know even more about the subject.  Finally, you must be able to incorporate those ideas into your own without focusing on the fact that you were bested in order to continue your education about the idea.

Chances are, though, that if you care very much about a subject, your position should not be boiled down to anything that could fit on a bumper sticker.  It should not come to buzz words, demagoguery, or rote quotations when you argue your point.  It should come down to wisdom born of knowledge, knowledge born of experience, experience born of research, and research leading you out of ignorance.  If it takes metaphors, such as thinking of LSI as a configuration space problem, that’s okay, but when Joe shoots you down and says “no, not really”, then be ready to slog through an explanation, read up on the ‘Net, or check it out through experimentation in order to build your new world view.