Californickation

Silicon Valley demographics

2011-12-04T20:47:00.000-08:00

The other day we were trying to find in the office anybody born in CA. The only person who was (and we do have a fair share of real American engineers) turned out to be a Chinese guy. I kid you not.

In other news, although I was the first one, now there are five people from a certain Far Eastern European country in the office.

All hail Geico

2011-11-27T12:31:00.000-08:00

Just noticed my car insurance for the next 6 months went down from $500 to $340.

out of stealth mode

2011-10-17T22:04:00.000-07:00

We have finally emerged out of stealth mode. It was interesting to observe the preparations from site design drafts to scheduling interviews with the press. There was even intrigue in exact timing - even though many people were notified in advance, all of them are supposed to keep the info to themselves until the moment our site goes live. One of our VCs managed to publish a blog in the morning (and my GOOG reader cached his posting in its entirety) and then they managed to unpublish it. But it seems to have been the only mistake in our publicity campaign so far. Let's see who will write about us tomorrow and how positive they will be.

This is the first time I got in early enough to receive stock options before B round and work for almost a year in a company with no real web site (I did not even know they still do it but enterprise software market is quite different from consumer-oriented one.). So far so good.

Unemployment? What unemployment?

2011-09-15T00:23:00.000-07:00

We have at least seven open positions in UI, platform, applications and devops. There are virtually no people available on the market. The last candidate to our platform team accepted the offer and then changed his mind a week or so before his first day. The rumor has it he was given annual bonus on the spot when the management at his current company heard about his plans. The most promising UI candidate was offered to work remotely and fly for a week to our SV office monthly. Have we reached a new permanent plateau or is it going to end in another 2008-style round of layoffs?

As a side note, how come in a 300M country there are not enough people knowing UI stuff? All that CSS/HTML/JavaScript is not exactly rocket science, students could do it. For 100K+ one would expect lines of people falling over each other to apply for a job .. even if on H1B.

The problem solving recipe of the year

2011-05-06T10:54:00.000-07:00

"Shoot them, bathe them, wrap them in cloth and throw them off a boat"

Curious power of written goals

2011-01-01T12:35:00.000-08:00

If you ever read or listened to anything on goal setting you know that in addition to typical SMART criteria there is one more cardinal rule - to always have your goals written down. The author usually says that only a tiny minority actually writes them down for some reason and the very same people tend to actually achieve their goals.

It just so happens that in late 2009 I felt very unsatisfied about my then current job. The people were nice, the benefits were decent but it did not feel worth of tribulation I had gone through by the time. I did have some idea of what I wanted even though it was quite stretching. So I went ahead and wrote down a few paragraphs. In particular, my dream was:

I want to work in a company:

- in Silicon Valley

- in an interesting/complex domain

- utilizing a few modern technologies (e.g. grid, DHT)

- having many high-quality engineers caring about their code

- with technological challenges of scale / concurrency / complexity / algorithms / AI

- where I could significantly stretch myself and make noticeable progress in my professional experience

- where I could learn a reasonable number of new technologies and practices and have exposure to smarter people

And the corresponding goal i set for 2010 was "look for unusual companies in Silicon Valley, apply every time, get interviewed in Silicon Valley at least 6 times" where by unusual I meant something more exciting than run-of-the-mill Java projects. And, at least in my experience, even here in the SFBA there are not so many such companies. I do not count GOOG and their ilk because I know I am not smart (or sufficiently grounded in CS) to get there and nowadays they are arguably too huge to be a comfortable environment anyway.

In total, I did end up interviewing at exactly six companies although two of them were in SF and not in SV proper (which incidentally only confirmed that SV jobs can be found only in SV and there's a huge difference between locations just 30 miles apart). A couple of them were very ego-bruising to fail. A couple more were interesting but I did not feel much affinity to the people or their products. In two large companies I got to the second round. Small companies were usually happy to never see me again after only one :) What I learned from this experience confirmed my hunch that it is all about "chemistry". Every single interview is different as are many if not all interviewers.

And it is virtually unpredictable what you will need - deep Java knowledge (the easiest part after nearly a decade with the platform), familiarity with algorithms (I usually forget a lot of the details in a year or two so it's a recurrent theme), experience with other JVM languages (e.g. scripting ones) or niche frameworks (e.g. Hadoop). The name of the game in software development is self-selecting groups and the only "nay" can seal your fate.

Nevertheless, the fifth company I went to is now my employer. They liked me a lot from the start even when I myself did not think much of them. And taking into account the caliber of their technical leaders and the complexity of the product I feel honored. Not to mention that now I am living the life I was so desperately eager to have. Aside from not having enough time anymore even to read all the blogs I am subscribed to I have not been as excited since 2003-2005 and even that high point of my career could soon become the second best in my life.

I am looking forward to one hell of a ride in 2011 .

Small world

2010-12-10T22:55:00.000-08:00

A few years ago when I was still waiting for my GC I was a consultant at a large bank. They had a vendor headquartered in Redwood City and once a few of us visited the vendor to discuss their next release. It was my first time in that part of the valley so I did not even know where exactly we were (near the exit from 92/San Mateo bridge to US101). And I enormously liked the office the vendor had. And felt so envious of people who could choose employers and were smart enough to work in the valley and in that beautiful new office.

So guess what - my new company is located in exactly the same office park in exactly the same kind of building. And if not for driving two hours daily I would be totally ecstatic. In a sense, driving is so much better than public transport (no freaks and minorities in one space with you and nothing stinks) but you also cannot spend that time productively by reading blogs. Anyway, I am awfully busy because the system will be large, almost nothing exists yet and I need to carve a niche for myself before other smart people come and take all the juicy components :)

You will always miss 100% of shots you don't take

2010-11-04T20:39:00.000-07:00

So just a few days later since my gloomy prediction I am about to pull the plug on my current job. After almost three years it feels pretty weird to think that I could move somewhere else in two weeks. I got quite used to the team and the code base although I am also a little tired of both :)

And I am pretty scared to be honest. I will lose all kinds of nice things such as matched 401K, HSA, 3 weeks of vacation, a very relaxed environment, and a 30-minute door-to-door commute I spend reading blogs. And all this for only a base salary (+10K which after taxes at best will be the same; though they promised to improve benefits late next year) and minimalistic health insurance.

But it will be in Silicon Valley (for the first time!), in a startup (but this time with very cool VCs and experiences founders who sold companies before), working on a platform in areas related to cloud and analytics and joining early enough that they have only a prototype in progress and a few technologies still not chosen.

In the nearest future my commute will amount to 90 miles a day which is as awful as it sounds - and more so because of heavy traffic so wasted time will by no means be 90M / 70MPH :( Some telecommute will be possible later but short of actually moving to SV there is no cure. The funny thing is I moved into my current apartment (which I like very much) just a month ago after a long time looking for a decent place in the area ..

But I feel so excited about this opportunity that I could probably accept it just for sheer fun of having engineering challenges with a few cool technologies in a green-field development.

Fasten your seat belts, it is going to be one helluva ride!

Post A-round startups in real life

2010-10-30T22:34:00.000-07:00

I have an offer from quite an interesting cloud-related startup in SV. Funded by A-list VCs. But with basically the same base salary but no 401K, disastrous medical/dental insurance (nasty copays, mostly 80% coverage only) it would be financially suicidal. Not to mention only 10 days of PTO. And 90 miles of driving a day unless I relocate after only a couple months in my current apartment. It seems to be an exorbitant price for some exposure to hot technologies. So my first real offer from SV is very likely to be declined for reasons that have nothing to do with engineering.

Java is so dead

2010-09-09T15:36:00.000-07:00

Lambdas in Java in late 2012 at best? So a stable JDK in 2013 and actual migration in 2014 or so? Even assuming they really manage to release something more or less on time. I cannot imagine smart people waiting for so long and the rest will likely follow. Now Scala, Clojure & Co will probably have a real chance to go mainstream so it could turn out to be good news after all.

Lies, damned lies, and bare metal

2010-09-03T15:40:00.000-07:00

A former colleague wrote about something I occasionally think about. In my view the post touches on at least three areas that could result in a holy war ("The need for bare metal programming skills", "The need for strong concurrent programming skills", "The right approach to concurrent programming"). What fascinates me is that I am not aware of a coherent narrative for at least some of them.

To begin with, the notion of bare metal is a blurred and moving target. From Cliff Click's detailed discussions to Joshua Bloch's higher-level recommendations there seems to be a wide-spread belief that contemporary hardware is virtually incomprehensible by most programmers. There are too many moving parts and indirection levels to have a mental model good enough for reliable predictions. And it's not just about high-level Java-like platforms.

As history of computing demonstrates, the hardware progress enables programming in terms of higher-level abstractions. And that pushes the border with bare metal higher with time. It's true for programming languages (consider how VM-based or interpreted/dynamic languages became practical in the last ten years) and for particular sub-fields such as concurrent programming (think about a recent surge of interest in Actors and STM).

Concurrent programming is a similarly gray area and also because of strong influence of hardware. On the one hand, shared memory and actors are just models and can be used to solve the same problems. On the other, software runs on real hardware and commodity hardware nowadays means you start from XCHG instruction and layer levels of abstractions on it. Be it Clojure or Scala, you are looking at java.util.concurrent (and, ultimately the very same CPU-level support of CAS) in disguise.

So with time some indirection levels become so low-level that only very few people have time/need/desire to look at. Today java.util.concurrent is the corner-stone and very few people bother even to look inside (for example, compare the number of people intimately familiar with JCiP with those fluent in AoMP). In a few years it might as well be Actors, if not STM/HTM. And then knowledge of j.u.c will count as low-level black magic. It's not easy to see when such a transition happens and adjust one's definition of "bare" or "low-level".

In addition, there are entire domains such as big data. People in the map-reduce land think in terms of [dozens of] servers, not threads. Actually, even in mainstream software not many are lucky to work with j.u.c-level abstractions and some even explicitly prefer even higher-level ones.

But in general, my experience confirms that serious usage of any technology implies comprehensive understanding of its design and some implementation details. As an example, you do not need to know about the GC to program in Java but if you actually do not you probably work on something trivial.

For complicated technologies it can be hard and take time and so one is necessarily limited and cannot be familiar with everything even in a particular language universe (just think about Java - from J2ME to Hadoop "and still counting"). At least in this day and age, most software comes with source code and so you can always dig deeper to learn the details.

Summer reading on actors

2010-06-15T10:30:00.000-07:00

For a popular topic, I do not see enough design discussions around Actors online. It could be difficult to learn about idioms or pitfalls without going deep into OTP or Akka. Off the top of my head I can think of only two interesting blogs:

James Iry wrote a fascinating series on why Actors are not a silver bullet for all things concurrent. He is such a polished author you should read his posts even if you don't care much about Scala/actors.
Somebody started recently a very informative blog with detailed examples of actor-based FP-style designs.

Object block initializor

2010-05-12T16:58:00.000-07:00

Object block initializor in Java is a code block in parenthesis executed before the class constructor. I always thought it was a bad idea because by default I expect all initialization code in a constructor. In addition, I have never seen such initializors used anywhere except for Java textbooks and interview questions.

Yesterday I for the first time read about a potentially useful way to utilize them. The idea is to save some typing (compare creation of b1 and b2):

Thunk

2010-05-05T14:43:00.000-07:00

I first learned about this approach to lazy initialization a few months ago but every time I look at its Java implementation I think it's a really cute piece of code. I wonder why my book on Singletons did not mentions it :)

iPad

2010-04-05T16:36:00.000-07:00

Today I played a little with one we got for work purposes) .Well, it looks characteristically slick. It's light. It seems to have a lot of screen real estate. It is also horrendously inconvenient for reading because its mirror-like screen reflects all the light sources behind you. So unless you sit in a dark corner good luck with any semblance of eye comfort.

They did not introduce any hardware innovation for navigation either, everything seems to be virtual. You have to touch the screen all the time even to scroll pages. Moreover, they use a UI so peculiar I could not even find a file browser (later I was told they do not have one) or any means of zooming a PDF document. Being an Apple product, I guess it will also have problems with downloading mp3s and ebooks by standard (e.g. USB-drive style) means.

Judging from how impractical the device is it will be a major sales success.

Second round questions

2010-04-01T14:42:00.000-07:00

The good news is that I consistently get invited to the second round in the valley, the bad news is that neither of the two companies I liked extended an offer. Well, at least I found the interviews intellectually stimulating :)

Imagine you are given a sequence of numbers such that all but one number are duplicated. Find that number.
Is it possible to go 1 mile south, 1 mile east, 1 mile north and return to the point of origin?
Explain the difference between Proxy and Delegate.
What's the difference in loading time between interface and abstract class?
Is there a SQL statement to eliminate duplication from a table without using temporary tables (subqueries are allowed) in one delete request?
Is there a one-line implementation of checking whether a number is even?
You have 8 balls of which one is different. Find the odd ball in no more than 3 weightings.

Whatever happened to programming

2010-03-24T14:43:00.000-07:00

In my blogroll I periodically observe how a lengthy posting triggers a wave of related discussions. The topic is usually rooted in reality and mildly controversial but no practical conclusion is reached. A few responses could be interesting now and them at least to compare with my own opinion. The most recent topic of this kind I encountered was WHTP.

Although I did not share the author's interest in games I certainly share his sentiment about what he referred to as programming and gluing libraries together. It is an emotionally-charged question with all kinds of repercussions for career perspectives and self-esteem. And I am inclined to believe that many people working in mainstream software development think about it once they reach a certain quality of life. Probably even the common heated debate on software architecture could be ultimately related to this perceived lack of hard-core engineering fun.

To begin with, a few comments rightly suggested that:

There is software and software. Nowadays computational devices are everywhere and the field is fragmented into multiple domains. Without clear focus on some of them the discussion makes little sense.
Many more people are employed in this line of business then there were twenty five years ago. So on average they must be less advanced.
Chances are many of those new positions created in these twenty years are in application-level software.
You probably consider different things to be cool when you are 18 or 30+.

Fundamentally, I think that most people who can relate to the original posting work in IT, not engineering. One can be successfully employed in IT without much of formal education because entry barriers are so low. Real software engineering requires a solid background in CS(AI, GOOG-scale data processing) or EE(embedded software, robotics) and similar scientific fields (for all things military).

There is also a down-to-Earth angle. Quite a few people were attracted to IT because of the lack of well-paid engineering positions elsewhere. This is how many people with degrees in anything from physics to mechanical engineering ended up in enterprise software. Only later they recognized that this area is qualitatively different there will be no return to actual engineering. On the positive side, they were able to immigrate to G7 countries or at least earn money pretty good by local standards (strangely enough, even in comparison with colleagues in rocket science).

I have thought a lot about it recently because I feel more and more that I got into a kind of ghetto. There is only so much fun you can have with util.concurrent, so to speak. And the longer I stay in this kind of development the slimmer my chances to work on something truly challenging are. I got a pay raise recently (4% is not as ridiculous if each of them is worth more than 1K), we are not overworked, my manager likes me, new Apple laptops are coming. Leaving now would likely require material sacrifices but I get this urge now and then.

I suspect that there are a few (sub)domains where even IT-like software development is at least moderately exciting. I have seen it in telecoms. I think finance is one of them (or is it just High-Frequency Trading?) but some might disagree. GOOG/LinkedIn/Facebook-like systems are likely to be fun because of the sheer scale (and could arguably be sufficiently algorithm-heavy to qualify as real engineering). But I am not quite sure how to get there. There is very little financial/telecoms development in the area. And I have not managed to get beyond second round interviews in really exciting companies in the valley.

A couple of life stories. A former colleague of mine was asked to develop some kind of DSP code (obviously in C) in the area of speech recognition/noise reduction. Last time I talked to him, he was reading math books and sleeping less because he had no relevant background (and after many years of Java). To me it sounded crazy (he is a bright guy but it takes some serious theoretical background) but exciting as hell. I also once came across a blog by a bright guy who found his way out(he wrote a few very interesting/inspirational posts on his journey from Java drudgery to cutting edge languages and academic engagements).

More Java interview fun

2010-03-03T16:03:00.000-08:00

I could not resist temptation of being interviewed by a company using GridGain and Hadoop in production. The position turned out to be somewhat different from the common set of Java engineer responsibilities but the interview did not.

Strangely enough for a heavily data-centric role (and they are pretty much competing with GOOG in local search on mobile devices) they did not care about algorithms whatsoever. Moreover, not all interviewers were even from the hiring team (sic!).

A) Imagine that you need to implement unoptimized but reliable char[] merge(char[] c1, char[] c2)so that, for example, [a,b] and [c,d,e] become [a,c,b,d,e]. So it's like merge-sort only easier. Basically, all it takes is

final int total = c1.length + c2.length;
while (k < total) {
if (i<c1.length) c[k++] = c1[i++];
if (j<c2.length) c[k++] = c1[j++];
}

and for the life of me I could not see what was wrong with my solution. Hint: imagine that both input array are of sizes larger than (Integer.MAX_VALUE / 2).

B) When a finally clause will not be executed? A tricky question because the answer is System.exit()

C) I am not sure whether it was funny or stupid but I could not remember precisely whether finalize() is public or even declared on the Object class (as opposed to, say, something weird like Serializable::writeObject). We also discussed: why object resurrection is dangerous (the method is called only once) and whether making "this" unavailable in this method would help (no because it would be impossible to access instance state and so clean anything at all).

D) A legitimate but not realistic questions about application of identity operator (frankly, I did not remember about the cache inside of Integer):
new String ("abc") == "abc"
String.valueOf("abc") == "abc"
Integer.valueOf("100") == Integer.valueOf("100")
Integer.valueOf("1000") == Integer.valueOf("1000")

E) A textbook example of interview questions I really hate (just imagine something like that in production code) - guess in which order it will print if you call new Parent() and new Child():

class Parent {
Parent() {
System.out.println(getFoo());
System.out.println(getBar());
}

public int getFoo() {
return foo;
}

public static int getBar() {
return bar;
}

private int foo = 1;

private static int bar = 2;

{
System.out.println("P-1");
}

static {
System.out.println("P-2");
}
}

class Child extends Parent {
Child() {
System.out.println(getFoo());
System.out.println(getBar());
}

public int getFoo() {
return foo;
}

public static int getBar() {
return bar;
}

private int foo = 10;

private static int bar = 20;

{
System.out.println("C-1");
}

static {
System.out.println("C-2");
}
}

Type-driven fun in Java

2010-02-18T11:40:00.000-08:00

A very interesting presentation on how a few powerful ideas could be used in real-life Java code. In particular, I liked what they call "type-driven development" and the notion of Thunk.

Incidentally, they also prove that it is not necessary for an early-stage startup to write a pile of ugly code which becomes instant legacy. Could it be because of their Stanford/GOOG pedigree?.

Sizing up Java employers

2010-02-15T12:01:00.000-08:00

While trying to understand what kind of company I would like to work for next I made another observation on employers in the Java land. At first glance they fall into three categories:

Typical Java shops doing something of little interest from the engineering point of view. Their job descriptions usually mention Spring and Hibernate. Frequently they also ask for either JEE or presentation tier technologies a-la JSP/Struts (IMO the latter is a particularly strong indicator you do not want to work for them).

Companies working on something more challenging or in more exciting domains. For me a good indicator is that they ask for more advanced/less known technologies such as Lucene, Hadoop or one of the numerous DHT implementations.

One-of-a-kind companies, GOOG-style. They do not list any particular technologies except for Java itself. They are explicitly after what they refer to as "bright people".

How much discrete math is enough?

2010-02-08T17:41:00.000-08:00

The more I am looking at FP, the more it looks like a rabbit hole. As I understand it there's a huge body of knowledge sharedamong most FP languages from Lisp to Scala (and this is why it's so confusing after C++/Java). And a lot of it is apparently based on actual math in contrast to OO hand-waiving (there's something called OO calculus but have you ever seen a book on OOP/OOD/OOA mentioning it?).

So I decided to skim through Types and Programming Languages to see what the egg-heads are talking about on FP blogs. And also a little bit of Lambda-calculus and Combinators just to double-check how much I did not understand :) Now the question is how deep I should go. I reckon learning a few concepts from Type Theory and Category Theory won't hurt. But I really doubt it is of much practical use unless one is planning to study Haskell. Which in itself is a questionable idea when there are so many more practical things from Scala to concurrent data structures. After all, I am not aware of a single company which would hire me just for knowledge of an FP language.

I should probably spend not more than a month looking at those theories and return to at least marginally useful things such as Scala or Erlang. And I definitely must peruse a few volumes on algorithms and data structures. In my experience that is the only prerequisite for joining a brand-name SV company.

New mp3 player

2010-02-05T15:47:00.000-08:00

After more research I found a model I liked enough. After a couple months I still like it. For under $110 I got 8GB built-in and another 16GB on a microSD card. It's extremely light, has tolerable navigation and buttons which are reasonably difficult to press by chance. What is really important is that it remembers last offset for audio book files. So although the device lacks a few small things my old player can do (e.g. playing sound while rewinding or showing how much time left for the currently playing file) I can live with it.

Interviews in Silicon Valley

2010-02-04T17:15:00.000-08:00

I am not actively looking for a new job. Most of the job ads I see are about positions only marginally different from what I have got already. Nevertheless I have a dream of working on something really difficult and interesting. After all this was the whole point of moving to CA (to be honest occasionally I think it was a mistake, my former colleagues who went to NYC or London to work on financial software have achieved more career progress).

It just so happened that in the last two month I came across a couple interesting opportunities. They were definitely not typical run-of-the-mill Java positions and so I decided to try my luck. I believe the two companies were very representative of the employers a smart person can find in the area. So although I failed to score a win with either I learned something about interviewing in cool places.

To begin with, I should mention that phone screening with both companies was quite different from face-to-face interviews. On the phone they were asking fairly common questions directly related to actual Java development: Singltetons, equals/hashCode and their ilk. Nothing you do not know after many years of Java development. The hiring manager from the large company asked me about what I later found was selection problem. By that time I had not read algorithms 101 books for quite some time so I did not remember the quicksort-like approach. But whatever balanced search tree gibberish I suggested did the trick.

The first round with the large company went smoothly. I spent quality time with three guys (a Ukrainian, a Philippino and an American). They asked me very reasonalbe questions - reverse a singly-linked list, implement the Map interface with some non-standard semantics, design a logging system. As it was quite in line with normal Java development I was able to convincingly answer most of their questions.

The second round (with two Indians and an American) was much worse and I am not even quite sure why. I failed to come up with a O(1) algorithm for a LRU cache on my own. Stupid me, we even have such implementation in our code base. With hints from the interviewers I finally understood how to do it with a hash table and a list. Doubly-linked list operations with all the edge cases did not look impressive on the white board. I was even worse designing an API for chess game. Two moderately uncomplicated questions, two uncertain answers. I felt pretty ashamed.

But at least I could observe a really cool Valley company form the inside. Every time I remember their office I want to kick myself for failing. They have a gym and showers on the first floor. They have a large kitchen full of fruits, protein bars, pre-packaged meals. They serve lunches. Not to mention the fact they are cash-flow positive, among top web companies and use all kinds of cutting edge technologies (but the crazy thing is they are running on JDK5 in production because many of their engineers use Apple laptops and something is wrong with JDK6 on Apple).

At a local Scala meetup I heard about a startup doing things I found exciting. So this is how I was invited for an interview with Standford graduates and former GOOG employees :) I knew it would be painful and probably humiliatingly so :) but hey, they invited me after all. And now I know how a Palo Alto barn (as in "large garage") looks like.

The first guy asked me to implement a thread-safe iterator (we agreed on something similar to how it is done in the Collections framework - with a counter and CME). and the corresponding unit test. His laptop had Eclipse instead of Idea and so it was incredibly painful to type anything at all. Unless you try to use an IDE you have no idea of you won't believe. On the positive side I wrote a bunch of concurrency-related unit tests and I am reasonably familiar with JCiP to not sweat much about such tasks.

The second guy asked me to implement addition and subtraction in a language with classes but no primitive types or conditionals. After a few hints I understood how to do it but I clearly I was failing. The third guy asked me about implementing a thread-safe cache, problems with double-checked locking and calculating some coefficient of a tree by traversing the tree recursively. I felt I answered most of his questions. Nevertheless when I came home the email saying a polite no was already in my mailbox.

So the conclusion is that exciting companies are pretty serious about interviewing and their questions will be somewhat unpredictable. The old suggestion to learn algorithms is definitely correct but it's not enough without some practice. I am planning to hit the books and see how much it helps in a few months.

Life beyond OO and Java

2010-02-02T13:40:00.000-08:00

The more I learn about FP, the more I feel like I have been misled and betrayed by the OO movement in general and its Java subset in particular. As an example, consider this post on pattern matching in Java (do not miss the comments). Not a single OO/DP book I have read in ten years ever referred to anything like that even as a side-note. Although the real culprit is the Sun team who knew the truth but still came up with a brain-dead language.

On the other hand, that's just my ignorance. With no CS background such books as "Types and programming languages" are a revelation to me. The deep relations between logic/math/CS and even lambda calculus itself are not exactly something I heard of in my youth. It turns out there was a very good reason why Java felt so powerless when I was moving to it from C++ and STL. It's a really scary thought that for years I could be so hooked on OO and serious about Java. Incidentally, I also found a very relevant discussion in comments to a developer v programmer post.

Probably I should not have been surprised by how unimpressed my interviewers were during a couple recent interviews in Palo Alto and Mountain View. Java/DP/OO/etc mastery is merely a necessary precondition to be invited for an interview but it will take much more to join a real SV company.

Stress tolerance training

2010-01-14T16:23:00.000-08:00

I have apparently failed the second round interview (not enough quick thinking on my feet I guess), had a root canal fixed and got a request from the USCIS to be present during my wife's AOS interview (and bring a pile of paperwork including a written explanation why I do not work for the original petitioner anymore). All in under two weeks of 2010.

Meaningful approach to interviewing?

2010-01-06T16:25:00.000-08:00

My current company is still looking for a Sr. Java developer in SF and I am usually one of the 1st round interviewers. Naturally, it makes me ponder on the optimal approach to interviewing. Recently I started noticing that on more and more topics my real answer is "it depends". It also frequently requires we agree on some conceptual model. I do not believe in the universal best answer to an important question but I also recognize that in real life any progress is possible only when one actually chooses and pursues one of the possible answers.

Interviewing is not any different. In my life I have witnessed at least five distinctly different interview types:

No real questions - you chat with a few members of the team but they do not test you. You tell them about your background, ask about their system and answer a couple trivial technical questions. Strangely enough, I worked almost five years for two companies that hired me in this style. Although I heard some people think it's a red flag (as in "Do you really think they will have sufficiently bright people with such poor screening?") one of those two was the best company in my eleven years. And yes, it happens in the US too.

Outsourcing-style - an endless stream of extremely low-level technical questions on particular technologies and APIs. RDBMS isolation levels, the servlet lifecycle, the methods on EJB home/remote interfaces, JSP scopes. A design pattern for them is always one from the GOF book or Sun J2EE patterns guidelines. Characteristically, they do not care about your design skills or code quality and really like J2EE-style.

CS undergraduate-style - in my opinion a questions such as "implement a RW-lock on the whiteboard" is a good example. Surely enough, real-life locks have nothing to do with whatever you will come up with in 15 minutes and, unless they prepared specifically, quite a few people will probably not be able to remember enough about textbook.

Common sense-style - reverse a list. Suggest a design for logging system. Implement a Map with additional non-standard requirements. You can answer them on the basis of real-life experience or with some thinking of the kind you do day-by-day at work.

GOOG-style (actually, I read it was MSFT who originated it) - CS undergraduate on steroids with a puzzle or two to load.

Personally, I was not happy during some of those interviews, especially the ones I knew I was blowing badly :) Incidentally, I also learned a lot from them. But when I have only an hour with a candidate I am not quite sure which approach to follow.

From what I see, a typical Java-centered company is either a J2EE enterprise or a place where they care about util.concurrent. Amusingly, this seems to be a very clear dichotomy. In both cases, there is a pile of [de facto] standard APIs and frameworks which take considerable time to master. And people who gravitate to one of the two types tend to be rather apprehensive if not ignorant of the other. I am also inclined to think that very little of even basic algoritms&data structures material is used in those companies. At least not deeper than TreeMap v HashMap and such. So asking more theoretical questions seems almost unfair (and not justified by actual day-by-day development needs).

As an example, today I interviewed another "Sr. Java" engineer. He did not know exactly what the volatile keyword meant (wtf? after all it's a keyword and not some obscure optional library). He was immediately confused with "try { foo(); return 1; } catch (Exception e) { return 2; } finally { return 3; }". And so on. He stated Hibernate experience in his resume but he actually meant "experience in calling Hibernate via JPA". Obviously he did not know about caching capabilities of Hibernate or mapping of OO models on relational schemas although it takes one book and a weekend to learn about. Frankly, I cannot imagine how a senior-level engineer could be so ignorant. After all, you learn by osmosis at work and glean questions from interviews throughout your career.

But what's the point of discussing, say, software architecture with people who fail simple questions relevant to daily coding? In a language universally regarded as outdated and extremely simple? Someone knowledgeable of actual CS must be pretty comfortable with basic hashCode/equals/HashMap stuff. Could I be gratuitously harsh on candidates because of some misplaced self-righteousness? Or I am right in that people who are too lazy to read half a dozen books before an interview do not deserve much? Or is my current company so unappealing that only losers apply?

Real life and FP

2009-12-30T13:31:00.000-08:00

An interesting and well articulated position on functional programming in the real world from someone with a nicely mixed background. Although his emphasis on game development could be less relevant for more typical software.

If I move to SV

2009-12-22T09:58:00.000-08:00

I was invited to the 2nd round interview on the Peninsula. Assuming it will be comparable to the first round and phone screening I will probably have an offer in mid Jan. And if I accept it I doubt I will tolerate a 60-mile commute for longer than a month or so. From what I see it should be possible to rent a 1bdr apartment for under 1.5K in the area.

Strangely enough, it looks like the major hassle would be changing post address. To begin with, tax return-related papers will start arriving in early Feb. It would be really unfortunate to miss on of the numerous W2s/1099INTs/etc. I am not quite sure how reliable mail forwarding offered by the USPS is.

In addition, there is a seemingly endless list of organizations to notify about my relocation. The USCIS, the DMV, the car insurer, all the banks where I have checking or savings accounts, the previous employer (for W2 for 2009), the company hosting my IRA, the company hosting my HSA, the postal service (to set up mail forwarding), the university my wife graduated from (to receive 1098-T for 2009), the company she had internship with (for W2 for 2009), the internet provider, PG&E. Probably: my current medical insurance company, the car dealership I bought my car from. And I am pretty sure I will come up with more ..

Interestingly enough, renting a 1 bedroom apartment in the Mountain View/Sunnyvale area is only marginally more expensive.

SV

2009-12-21T12:18:00.000-08:00

So I went to Mountain View last Fri. It's a fascinating area with anything from googleplex to computer history museum just around the corner. And it looks more or less like a park, very different from filthy stone jungles of SF. The company and its headquarters feel so vibrant it made me want to join them on the spot (not to mention their impressive cafeteria). I really feel like moving to SV this year one way or another.

Surprisingly enough, the interview was not particularly challenging. Reversing a list and a couple general design discussions (e.g. about a peculiar Map implementation). I guess the interviewing team was representative of SV demographics - Russian/Indian/Filipino/American engineers, American manager and HR. Let's see if I get invited to the second round.

Or, and one more thing. Driving (even almost 60 miles one way) feels so much better than public transport. Although losing 1.5 hours of daily reading on BART would be hard to compensate.

Looking for an MP3 player

2009-12-21T09:54:00.000-08:00

For a new car I also need a new MP3 player. The other day I checked what's on the market and I was quite disappointed. There is literally no player with exactly what I need regardless of price. Moreover, most of them are equipped with unnecessary features. If not for its capacity, I would very much prefer my still running iRiver 380T (128MB) to anything I found on amazon.

To begin with, there are arguably just a handful of features one needs. The ability to copy you mp3 files from the hard disk to the player following the same directory structure, preferably via the "USB mass storage" route rather than a proprietary UI. The ability to bookmark multiple files to return to the last read offset in a few audio books. A lot of capacity (arguably, at least 24-32Gb). To my surprise, even presumably most advanced (and expensive) players including ipods fail on the first two accounts.

In addition, I could not find a single modern player without a huge screen. The best I saw was something like the ipod classic but even it has a color screen which is likely to hurt power consumption and price. At least APPL produces players of impressive capacity.

I almost decided to shell out $230 for the ipod (despite that stupid itunes UI they require) but I found that they only allow bookmarking in audio books in some obscure formats. Moreover, even after reading documentation I am not sure if they can just copy my MP3 directories or I would need to create some idiotic playlists. WTF? I mean if you have 160GB of disk space you could automatically bookmark every single file - 4 bytes per file does not sound particularly burdensome.

So it's Corolla

2009-12-14T09:46:00.000-08:00

After almost three years here I have finally bought my first American car. The funny thing, I did not have time to go anywhere because I am preparing for an interview. So I just drove it home. Chances are I will drive the first hundred of miles traveling to the Valley for the interview.

While test driving I did not notice anything unusual about the car. I would even say that it felt quite natural. I was accelerating and changing lanes but I was not able to observe whatever is supposed to be chronically bad about steering and handling. I guess it would take many more miles to sort it out. Plastic looked rather cheap but at least there is a real clock on 2010 located inconveniently low. Actually I like the way controls and meters are located but I am less excited about the yellow night lightning. Even Accord looks better at night with appealing blue lights.

I've been planning to buy a car since early summer but I was not sure which one was optimal. Corolla is believed to have the best MPG, standard stability control (still not sure what exactly it does) and something wrong with power steering. Civic seems to have comparable MPG, less road noise protection and lack stability control-like features completely. And the Civic seems to have been even more reliable recently than the Corolla.

I drove 1K miles (including about 400 miles at 75MPH) in UT in Corolla S and did not notice anything unusual. But they say 16-inch wheels on S behave differently from 15-inch ones on LE. Unfortunately renting a Civic is not an option for some reason so a test drive is the only way to ride in it.

The true value reports from Edmunds show that Civic is about $2K more expensive (i.e. $3K after insane CA sales tax). So the OTD price of Civic is at least $20K while probably about $17K for Corolla with current rebates. And Dec is the last month to buy a car if I want to save a few crumbles from federal income tax.

The best quotes I received from local dealerships were:

Corolla LE 2010 (with CC, RKE, floor mats): $14,998.00
Civic LX 2010 (sedan, AT): $17,459.63

So pretty much as I expected, the difference is about $2K+. Interestingly, the OTD price for Civic would be $19,429.48 (1,620.10 sales tax, 294.75 lic./gov. fees, 55.00 doc. fee). For Corolla it would be $16,846.15 (1,466 sales tax, 331 lic./gov. fees, 50.00 on doc. fee). Speaking of additional fees, I found a nice write-up of things added to base price.

I also learned my credit score (for some reason the dealership checked it). With a credit history of about 2.5 years, 2 credit cards and about 10K of available credit the score is 713.

External motivation

2009-12-10T14:49:00.000-08:00

Curiously enough, they responded with an invitation to a real on-site interview. I guess I have no choice but to re-learn CS 101 in a week or less. Algorithms in Java should do the trick (and I definitely prefer it to Cormen's Introduction to Algorithms for practical purposes).

And I guess I'd better buy a car within a week although renting one to go there could be an option. Strangely enough, there is no reasonable way to commute from the East Bay to the Valley by public transport.

2009-12-09T17:29:00.000-08:00

As I understand, big and famous software companies have a pretty different style of interviewing than typical employers in the Java land. They virtually do not care about one's expertise in mundane subjects such as particular languages, frameworks and other acronyms you can find in a typical resume. Sure, they will ask you a few Java questions to warm up but the real meat will invariably be CS 101.

Personally, I find it very challenging. I am not a CS graduate so I tend to quickly forget the details of algorithms I read about. As most probably most Java developers I do not need them on the daily basis. So when I decide to refresh my knowledge of this area, in a year I do not remember much. I heard at least some people with their eyes set on GOOG hit the books for a few months before applying.

In general, I agree that the best HR strategy is to find and hire smart and motivated people. And arguably one's prowess with algorithms and puzzles is very likely to correlate with IQ. I believe that for reasonably smart people all the good design skills/best software engineering practices/TDD/OOA&D/UML/Patterns/etc are just a mandatory part of their skill set. It's pretty much a commodity based on common sense and some time in the industry. This explains why the best employers use different metrics.

On the other hand, not so many people are lucky to deal with stuff more challenging than util.concurrent and basic caching. Which means that there's a significant gap between what one actually practices/knows/uses in normal development and what is required to pass an interview in a cool company. I am not quite sure how to reconcile these two things.

Trying to learn by heart a couple of books on algorithms and data structures sounds like a plan but it would take a lot of time otherwise allocated to new things such as Scala or Hadoop. And if I remember my college days correctly, most of that knowledge will have a rather unimpressive half-life. Moreover, judging from job ads most companies ask specifically for experience with particular technologies (be it Lucene or J2EE) and could not care less about your ability to implement binary tree rebalancing in ten minutes. And from what I know, they will even pay you comparable money.

Anyway, today I had a phone interview with such a famous company from the Valley. We started from a discussion of singletons. Curiously enough, I correctly came up with three ways to get multiple instances of a Singleton in the same JVM address space although I remembered only one off the top of my head. But when we progressed to the algorithmic section I did at best averagely. Not a big surprise but it hurt to demonstrate so much ineptitude. I think more and more about finding a more challenging employer and relocating to the Valley. So probably I should postpone trifling with new and interesting stuff and hit the books for some time.

Seniors among Java engineers

2009-12-04T12:09:00.000-08:00

If you were to trust what you read on the Net you would think Java is a brain-dead language and people who develop in Java are the alternatively gifted among software engineers. While the former is a multidimensional question the latter could make you nervous if you've been in the Java land for many years. Allegedly, in some circles a decade of predominantly Java experience raises a red flag.

I am inclined to think that Java is mostly infamous for its enterprise branch. Surely once fashionable J2EE technologies are good for outsourcing but the damage they inflict on developers is very real. But if you ask me, I'd rather do EJB2 than any kind of web UI from JSP to ajax. Luckily I have not had to deal with either for a long time.

Occasionally I participate in interviewing potential future coworkers. And some of them position themselves as Sr Java engineers and even have CS major backgrounds. You would think that in the Bay Area developers are likely to be a uniformly enlighten group. After all, this is arguably the most expensive place to hire and an international magnet for software engineers.

Not really if my experience is representative. I have interviewed and even worked here with people who demonstrated almost appalling lack of professional skills (e.g. decent OO design with proper decoupling and meaningful responsibility assignment) or relatively basic knowledge (the role of hashCode/equals in the HashMap context, anyone? optimistic/pessimistic locking strategies? ACID? mapping of class hierarchies on relational DB tables?). On a few occasions after a decade or two in the industry.

Or is it just me? Probably I am supposed to ask candidates to implement a RW lock (unless your name is Doug or Brian I'd rather know you are intimately familiar with the util.concurrent package) or recursive tree scanning? But those look like more advanced concepts to me than the difference between volatile and synchronized in protecting a long variable. And it's not like in their defense they could show years of non-OO (although I am pretty sure FP folks are into good design, only by different means) or non-Java/C++ experience (then again, what makes him a Sr Java engineer?).

Could it be the case that Java is merely so dominant that it covers wildly different domains (e.g. util.concurrent/NIO v J2EE; telecoms or financial middleware v a web UI for DB access) and we do not have specific terms for people with the corresponding expertise?

CAP for the rest of us

2009-12-01T12:12:00.000-08:00

A nice two-part discussion of the CA P theorem with practical examples.

I did not call you stupid, I called you a Democrat

2009-11-09T12:35:00.000-08:00

Last year I had some hopes for the impact of the last recession on the Socialist Republic of California. Even basic things like sending the Mexicans home and the Democrats packing, cutting free goodies for the alternatively gifted (e.g. welfare, section 8, rent control in SF, mortgage modifications and moratoriums) would do miracles.

Apparently we are nowhere even remotely close to addressing real issues. As the latest illustration shows, the unwashed masses are happy with both the America-hating communists from the White House and their local accomplices. The other week CA elected another Democratic congressman. All the while lamenting the state's future.

If only Silicon Valley were located somewhere else :(

Some people are living the future today

2009-08-20T21:49:00.000-07:00

This is the first pubclicly available long presentation by Click Cliff on their HW to the best of my knowledge. They seem to be both living in the future and doing hardcore CS/EE. I am surprised Doug Lea does not work for them :)

Best sticky note presentations on concurrency ever

2009-08-13T18:01:00.000-07:00

I stumbled upon a few very impressive, useful and uniquely designed presentations . They discuss a few classical but very realistic concurrency problems using a distinct visual and didactic approach. Frankly, I have never seen anything like that, the author seems to be in a league of his own.

The only inconvenience is that you need a connection all the time because the system hosting those presentations does not cache all the images.

CTM for dummies

2009-06-23T10:38:00.000-07:00

I heard a lot about Concepts, Techniques, and Models of Computer Programming but I have not read it yet. It turns out there's a 30-page paper by one of its authors which crams most of the book into a high-level overview. I was instantly fascinated when I started reading it. It's short enough to read while commuting and it puts into perspective most major languages and concepts and paradigms they implement.

Multicore prorgamming lectures from MIT

2009-06-23T10:30:00.000-07:00

Yesterday I found this newly uploaded course. The content seems to be very interesting at least from hardware and conceptual points of view.

How to leak Linux pipes from Java code

2009-06-19T15:56:00.000-07:00

Fixing a bug in legacy code I found a moderately counter-intuitive rule for dealing with Process instances returned by Runtime when one executes a process. It turns out that the three standard streams associated with each instance must be closed regarless of whether you use them anywhere in your code.

By default I expect that unless I explicitly open something there is no need to close it manually. Moreover, I double-checked javadocs and Java Programming Language and I have not found any related warning. On the other hand as it happens on the boundary between java world and OS it is probably natural to expect very strict requirements for disposing of unused resources. At least it was not me who wrote that code in the first place :)

An easy way to check if pipes are leaked is to run something like "/usr/sbin/lsof | grep pipe | grep java" and see if the number is growing. In my experience if you get a "too many open files" error it's likely to be either about leaking pipes or sockets (if you can see an unreasonable number of sockets in CLOSE_WAIT state in netstat output).

A-Z of Programming Languages

2009-06-16T13:31:00.000-07:00

A nice series of interviews with 21 language designers from Ada to Erlang.

Glimpses of Scala future

2009-06-05T10:29:00.000-07:00

Yesterday we had another BASE meeting. This one was particularly interested because Martin himself attended it. It was intriguing to observe an international celebrity sitting 6 feet from him. Curiously enough his laptop was an ordinary Windows machine, not an Apple.

I was much more impressed with a few remarks concerning future Scala plans than by discussion of changes forthcoming in 2.8 (mostly re-designed Collections, support for continuations and specialization of genertics with annotations). To begin with, Doug Lea was reported to be so frustrated with lack of Java progress that he was going to participate in Scala concurrency library. If it actually happens I think it will be a clear indication that Java is beyong the point of no return.

Another prominent engineer, Josh Bloch, was said to be going to seriously look at Scala. It would be interesting to hear his opinion, after all there is no guarantee that Scala will be the next Java and Josh is surely among top 3 people qualified to judge Java-related matters.

In general, Martin's vision is to make Scala the language of choice for concurrent and scalable systems on the JVM within next five years. I would say that the time frame for choosing Java's successor will be shorter, probably something like a year or two. If Scala manages to have a stable and performant implementation of FP/Actors/Collections (and limit the number of inconsequential syntax changes) and incorporate Doug Lea's concurrent abstractions omitted from Java7 it will position itself perfectly in the future JVM space.

They knew it but they still ended up with Java as we know it

2009-05-23T10:41:00.000-07:00

Ten years ago I did not know who Guy Steele was or that I would spend years programming in Java. Seven years ago I thought that a language without operator overloading and templates was an ugly toy. A few years later I got so used to it I mostly stopped worrying.

It turns out Java authors were not morons. Probably they thought the users of their language were. Or they just could not imagine that a language for programming refrigerators (or was it set top boxes?) would be of any wider use. What is painful to learn in hindsight is that they knew about all the good suff , liked it a lot themselves but somehow chose to ignore it.

Back to basics

2009-05-22T14:57:00.000-07:00

When I started reading about Erlang and Scala last year I felt rather uncomfortable. Many small syntax details looked rather odd after years of C++/Java development and made little sense. I remember how learning Java I just had to remember what is missing in it in comparison with C++ (pretty much everyting advanced - templates, operator overloading, STL). This time it was different and even vague recollections of overloaded parenthesis were not sufficient to easily grasp a functional language.

With pretty much everyone online saying that Java is either dead or at least brain-dead and looking forward to multilanguage programming future I decided to dig slightly deeper. So recently I found myself learning basic Lisp/Scheme. Conveniently enough, there are MIT and Berkeley video courses on YouTube based on the legendary SICP book. To me mixing medias helps a lot.

I am not sure so far how significant the influence of all this will be on my work. Certain qualities such as immutability had been appreciated long before Java became popular (const int* const i ; anyone?). And I am still struggling with suspicion that lambdas and classes are merely two sides of the same concept. After all lambdas are not "functions", they are associated with frames which actually represent hidden state and not merely a bunch of machine instructions.

Not being a CS major I missed early indoctrination and so now I am re-discovering for myself things which are probably self-eveident to better educated people. FP in general, and Lisp in particular seem to be very common in different CS fields. So much so, that most AI books and many introductory CS textbooks are written with Lisp examples only.

With my EE background it is not entirely clear whether it means that I have spent my professional life in some low-life corners of the software field or that those nebulous areas are of very limited usefulness in real life. It's not like I see many positions for Lisp/Scala/Erlang programmers or AI developers even here in the SFBA. Most startups I am aware of are not working on robotics or something like that either.

A relatively unexpected observation is that it is not that difficult to learn the basics of a new language. The core of Lisp is both small and excitingly related to CS. My problems with Scala/Erlang seem to have been caused exclusively by my total FP illiteracy. So the motto of "a new language a year" makes more sense to me now.

M&A

2009-05-21T10:01:00.000-07:00

Probably as a sign of weakening VC climate we have been acquired. By a publicly traded company and they even promised to give us their stocks. On the negative side, they have significantly more expensive health insurance (especially in comparison with HSA we used to have) and absolutely atrocious choice of 401K funds. At least there could be some interesting tech stuff to do related to scalability/cloudness because of their extensive network infrustructure. We'll see.

This is the third startup I have worked for so far. The first one, a few years ago grew to a self-sustaining business. The second went down is style. This one got acquired. Judging from the trend, the next startup I will work for is likely to fail :)

Big4 on YouTube

2009-04-16T17:15:00.000-07:00

Recently I noticed that the Big 4 (i.e. MIT/Stanford/Berkeley/CMU) shared a number of their introductory courses online. At first glance, only Stanford and Berkeley actually have a few CS-related ones. MIT shared mostly general engineering/math courses while CMU, the home of robotics and software engineering, has virtually nothing online.

I am watching two courses currently to get some idea of Lisp/Scheme and how the best and the brightest are inculcated true meaning of CS in general :)

Stanford

Programming paradigms - a very nice review of a few languages (from Assembly to Python) including four or five lectures on Scheme. The lecturer is very energetic and gives an impression of someone who has real life experience.

Berkeley

CS61A Structure and interpretation of computer science - more of a CS course using a Lisp dialect than a language course which arguably makes it even better. The lecturer here is less impressive and at times seems to be too old (incidentally, the equipment is also less shiny than at Stanford .. is there some kind of correlation?). There's also CS61B but it seems to be based on Java (?!) so it is less likely to be of significant interest.

MIT

MIT 6.001 Structure and interpretation of computer programs - vintage '86 edition, narrated by Messrs. Abelson and Sussman themselves.

In addition, there are a couple more courses on subjects such as robotics and machine learning but those have less immediate applicability. So I encourage you to browse their offerings.

Book marketing for real programmers

2009-03-15T19:13:00.000-07:00

ForkJoinWorkerThread.java :

"... If you are unfamiliar with them, you probably want to read Herlihy and Shavit's book "The Art of Multiprocessor programming" ..."

The book seems to be JCiP on steroids (would you expect less from something Doug Lea himself refers to? :) ) with 100+ pages on theory (pretty standard topics with nice illustrations, Java-friendly examples and quite detailed explanations) and 500 pages of everything you always wanted to ask Doug about (personally, I would not expect to understand most of the answers anyway, at least before reading this book).

It took me five cover-to-cover readings of JCiP (guess who is the slow one in my family? :) ) to get sufficiently comfortable with the subject. And I still occasionally re-read parts of it. AoMP addresses much harder questions although is probably less practical unless you are writing your own j.u.c or want to understand how the real one works.

Scala Lift book

2009-03-06T14:25:00.000-08:00

So it turns out there is a free book on the Scala web framework written by Lift developers. It is 255 pages long in pdf so it is not a toy. The bad news is that for some reason the book is provided as a bunch of LaTeX-like files. I had to downlosd a 100MB (sic!) installer for a converter to pdf. WTF?!

[Edited on Mar 09]

I skimmed through the book on Fri. It is a really nice introduction covering all the basic things (from Maven 101 and generating an empty Lift project to O/RM and AJAX support) so I believe the Lift guys should be more vocal about it (not to mention sharing a version in some user-friendly format). It definitely answered my beginner's questions about Lift.

On the negative side, the Lift framework is work in progress and so the book marks up some chapters as "provisional/partly to be deprecated". I was particularly confused with the chapter on OR/M because they mostly cover the original approach which seems to be deprecated. By no means this is something to blame authors for and I hope they will ultimately synchronize the book with the Lift roadmap.

I am not really into UI/presentation tier so I am in no position to judge the Lift framework. For some reason I was not impressed with what I saw but do not trust me in this respect. In addition, taking into account how many XML manipulations are hidden under the hood (and Scala's ability to make XML processing look cheaper) I am worried about performance.

One area I want to research further is their approach to continuations. As I understand they rely on the container to provide basic infrastructure and so no magic should be expected from, say, Tomcat5 deployments. A related questions could be COMET support with Actors because both are quite new to me. I think those two areas could be of general interest from the usage of the Scala language perspective as well.

In conclusion, I applaud the authors for a very useful resource for people interested in Scala.

Small language changes for JDK7

2009-03-02T15:27:00.000-08:00

I am a thoroughly conservative guy and quite naturally tend to be a late adopter even if I am aware of a particular new technology in general. Watching a panel on future of Java on infoq.com a couple of years ago I was quite surprised by people of Joshua Bloch caliber discussing Java as a probably mature/declining language.

Back then I expected that somehow that language changes of Java5-magnitude would be added with each new release. I believe that only in Java5 the language was getting some of the C++ power it was originally deprived of (e.g. STL and all the features enabling it). So it felt as Sun finally decided to get serious about the language and stop dumbing it down to almost VB level.

Surely enough the titans were right and saw it coming even back then. In contrast to the JVM (as in garbage collectors or contested monitor performance) Java does not seem to have meaningful progress anymore. Syntactic sugar just is not worth it in my opinion. And Doug Lea seems to be the only one pushing Java's limits.

Things like catch blocks for multiple exceptions do not bring much to the table, not to mention all the legacy code which is not likely to be migrated. And to have mixed styles in the same code base is even worse than to have gratuitious blocks in the first place. As a creature of habit, I do not even notice such things anymore, I code them on auto-pilot.

Last year I was reading the Erlang book and got so confused with FP idioms that I had to digress and read the Scala book to better grasp what all the FP hype was about. It did not convert me from OOP but definitely made me appreciate FP (and incidentally reminded me about STL which enabled all the FP-style fun with lists fifteen years ago).

And somewhere down that road I found myself thinking that I do not want Java to be changed dramatically anymore. An attempt to make Java more sophisticated would result is a limited version of, say, Scala. It would be a new language and all the existing code would immediately become non-idiomatic. Or in the worst case there would be just a few special cases hard-coded into the language instead of meta-protocol-like facilities.

So, if even I have this funny feeling that any potential Java upgrades (years ahead, and some people have not migrated even to 1.6) are not as appealing as a modern language such as Scala probably the tide is actually turning.

Programmer Competency Matrix

2009-02-27T14:08:00.000-08:00

It is not he first such matrix but it made me think a little. In a week I will be celebrating my first ten years in professional software development. And looking back I can only laugh at myself back then. It's been a long way from almost "Unable to find the average of numbers in an array" but log(n) (Level 3)-like requirements show that the next decade will probably make me feel something similar :)

WFH

2009-02-25T11:11:00.000-08:00

We are working from home for a few days because of moving to a different office (this time it will be 3 blocks from Market St istead of just accross the street from Montgomery St BART station). As was predictable, productivity is going through the roof. Few things can be as valuable as a quite space where it is possible to keep concentration for a long time (not to mention 2 hours spared from commuting).

I wish we routinely worked from home, say, 50% of time, especially in the middle of a sprint when requirements and design are already well understood and it is time to code.

Not even two months later

2009-01-30T12:05:00.000-08:00

.. and we are losing another three people. This time it is not just about auxiliary folks from support and QA, the least experienced developer is let go as well. So we are down to 13 from the peak of 20 which is a 35% cut. And we are moving to a new significantly less spacious and expensive office next month. And we do not even have water service anymore.

I would say it is not down to bare metal yet but it is getting pretty close. The board of directors think that the lucky head count of 13 is reasonable for the rest of 2009 but let's see what they think in a couple of months. Last December they thought the same about 16 people :)

Programming in Scala on Actors

2009-01-29T12:47:00.000-08:00

I am through my first reading of the book and somehow it seems to be less helpful than I hoped it would be. I do not think it is bad or merely repackaged online documentation but I guess I expected more from a book written by the language author. Sublime wisdom or something :)

I believe most people are interested in Scala because it is an OOP-friendly tool for learning practical FP and Actors. So I was particularly irritated by chapters on GUI (who cares?) or minute details on integration with Java (better to be online instead of wasting precious book space). On the positive side, my feeling is that FP is covered in the book adequately.

What truly disappointed me was the chapter on Actors. To begin with, why only 1 chapter on this exciting topic? It is not like best Actor practices are better known than FP among us mere mortal with mostly C++/java background. And in my opinion that event simulator example is actually a good illustration of what not to do.

What I wanted to see was a clear comparison with traditional concurrent designs. So some universally-known concurrent system (e.g. a network /web server) would not require any explanation what it is and would immediately conjure up many examples from java world. Some code snippets would be probably even useful in real life. Not only modelling circuitry won't, it actually complicates the discussion with essentially unrelated clutter.

In addition, it would be really useful to learn how all this Actor machinery actually works. Is it some native magic or just a Scala library? How does it compare with j.u.c-based abstractions, at least performance-wide? Not a single word.

As a matter of fact, there is a much more elucidating paper named Actors that Unify Threads and Events. It really helps to read it before you delve into the dark internals of Actor.scala. It turns out that some interesting (and somewhat dirty? or truly ingenious? exceptions for control flow ..) design decisions were made to make it work.

As for the Actor-related code itself, I found it quite messy. Makes me wonder if the root cause was extreme performance fine-tuning or certain sloppiness. There is even a recent post where an independent Scala enthusiast comes up with a more orderly design of Actor.scala (and provides a few more hints on the official design). Hopefully Martin&Co will listen to him.

Is Java relevant?

2009-01-11T09:18:00.000-08:00

My previous post made me think about my attitude towards Java. Even if it happens to be behind its pinnacle, is Java any good beyond paying my bills for years? If we know what Java gave this world probably we will be better equipped to decide when to move to a new language and how to judge it.

History teaches us that non-technical aspects matter more than what is habitually referred to as technical excellence. Which is actually difficult to define. I have a growing suspicion that what matters a lot is something I would call, for a lack of standard term, "age and experience demographics". The idea is that certain technologies are perceived to be cool by people entering the field. Those technologies in a sense create an imprint on that particular generation.

As an example, in the early nineties Assembly was for real men and C++ for real programmers. At least how I felt it back then under certain circumstances (no influence from CS-savvy people, and extremely limited literature in the absence of the Internet, most people using and targeting MSFT platform). I know that years before I discovered computers people had been writing in HEX code and considered Assembly to be unmanly. It very well may be that nowadays something like scripting/dynamic languages are all the rage among the young.

There must be also a heavy influence of experience resulting in some kind of "shifting interest curve". You need to be through a complete life cycle of a major language such as Java to be able to see that not everyting is new under the sun. As an example of a self-made developer, I have progressed through Pascal, C++, OOP, OOA&D, RUP, UML, Java, algoritms and data structures 101, development methodologies, software architecture, really understanding OOA&D, really understanding Java concurency, TDD (disclaimer: for simplicity sake the sequence was artificially serialized and stripped of many things).

Right now I value good analysis and design (assuming OOA&D; I am not aware of FP/dynamic language alternatives to it; and yes, I appreciate UML) and best software engineering practices more than a particular language. I like reading about algorithms much more than I did ten years ago. I am dabbling with FP in its safely diluted incarnation (Scala) and planning to look again at Erlang later next year. Who knows, probably in my next ten years I will be able to get interested and even like highbrow stuff such as LISP. The point is that in addition to normal technological progress each individual has another learning curve which includes things not affected directly by his current job.

And last but not least, not everyone majored in CS. As a matter of fact, I believe I have worked with more people from Engineering/Physics backgrounds than properly educated CS graduates. Naturally, the former were not indoctrinated in seemingly non-practical subjects such as LISP, FP or even the details of compiler design. Some of those are probably later picked in spare time but I would still expect a gap between originally CS and non-CS folks.

So, what do I like about Java? I happen to maintain an Apache module written in C and every time I fix a bug there I appreciate our warm and fuzzy Java world more. This juxtaposition elucidates good things about Java although by no means it is even remotely scientific or exhaustive. Disclamer: borders between Java the language, the JVM and Java ecosystem at large are blurred to reflect real life.

Java has a standard API (or a de facto standard API) for nearly everything. And innumerate open-source implementations of those.
Java has multiple open-source components available for anything from reliable messaging and IoC to web UI and Map/Reduce on serious scale.
We have a set of widespread mature technologies covering most important mundane areas (e.g. Maven/Spring/JUnit)
Java has a memory model and an amazing util.concurrent library. It was Doug Lea and not James Gosling who was my youth hero :) And Brian Goetz wrote really nice javadocs :)
Java has a decent Collections library
Java dominates the server side. Jobs are abundant in interesting domains from Finance to Telecommunications
Java promotes OOP and interface-based design. Not really design by contract though :(
Because Java is a dumbed-down language, it is easy to read (well, multi-screeen methods are still found out there but I guess you catch my drift). Though I have always missed overloaded operators and proper generics (enabling the STL library in C++).
Nowadays there are enough high-quality Java books
Without IntelliJ IDEA I would go out, shoot myself in the foot and die :)
Java does not look weird (presumably in comparison with the C branch of evolution)

Most of us would be able to continue this list for a long time. When I look at it the point seems to be indeed that it is not about Java the language. Libraries and the JVM sneak into the picture all the time. As a creature of habit I like Java because I am used to it but if a JVM language more sophisticated than Java comes along I would be all for it.

Harking back to Scala, the language looks smart to me (yes, I am enamored with C++ like complexity and things like covariant/contravariant type annotations attract me rather than scare away). I detest languages that restrict me (Java more than C++, Erlang more than Scala). I cannot imagine living with a language where I would not be able to do OOP even if I like the idea of trying out at least elements of FP. And Martin reminds me Bjarne more than others do :)

Is Java obsolete?

2009-01-09T13:57:00.000-08:00

This topic seems to be nearly obligatory nowadays in Java circles. Is Java a mature language about to enter its years of slow but inexorable decline? Can it be resuscitated? Should it be? Does anybody care? :)

Clearly, future will tell and there are more experienced people out there who can better articulate their predictions. As most people in my generation I moved to Java from C++ and from the very beginning I knew how many sophisticated features were missing in Java. Back then I was still learning the ropes so it is safe to say that Java is the first major language I have seen evolving from its early years. When I started using C++ it was already king of the castle . Although reading about its history was amusing I was too young to truly appreciate the concept of language life cycle.

Harking back to Java, I am more fascinated by historicity and sociological/psychological roots of programming language preferences and fashions. There are usually a few competing schools of thoughts in most technical areas. Thus, even if I tend to align with some of them I do not believe there is the only right answer. And I hear that quite often technical prowess does not always decide the fate of a technology. The longer I live the more I tend to see heavy influence of wetware on seemingly semi-scientific matter.

I remember talking to a colleague about it a couple of years ago. Under influence of the first(?) book on Java obsolescence I told him that we were not likely to write in Java in, say, ten years. I recollected the year 2000 when I myself was surprised why anybody cared about that obscure thing called Java. I was even more stunned when MSFT embarked upon cloning Java under a musical brand. Why anybody would want anything else if they already had C++?

Fast forward 8 years and more than once I have seen blogs saying that in a few years Java will be actively used only for "low-level, system development". The notion of "low-level" has changed a tad since the time I was young :) And to my personal taste, I'd rather stay in Java than do whatever will be classified as "high-level".

Lacking a CS degree I hardly heard about functional programming or LISP in my youth. And certainly never encountered such things in my professional life. When the recent FP fad started spreading through blogoshere I was mostly taken aback because I did not make much sense to me. And I was pretty irritated with the idea of learning a new language a year. Frankly, I still do not count SQL or Ant scrips as languages (unless implying DSLs) and I do not think a year of even full-time development is enough to learn a serious language (as my experience with C++ and Java tells me).

Actually what is meant by that suggestion is that there is apparently no general metamodel for features found in different languages. So there is no way to learn about a useful concept or idiom without actually spending a lot of time on learning to read languages one is unlikely to use in real life. Which sounds like wasting a lot of time to me. Actually, I prefer frameworks to languages. As an example, deep inside the Erlang runtime all those marvelous message queues and clustering features are actually implemented. I'd rather read a book on algorithms used in that implementation than learn Erlang simply to familiarize myself with those concepts. Alas, I have never seen any Erlang runtime design documentation.

I am also yet to find a paper clearly explaning how OOP and FP could be two sides of the same coin. As they intuitively seem to be (e.g. think about Erlang actors or modules as classes .. yes, strictly speaking it's far-fetched at best). Judging from the dearth of literature on FP design or UML analogs (or at least profiles) for FP people somehow manage to survive with methodologies originated in OOA&D. Which implies certain affinity.

Anyway, I am reading the first book on Scala. It's fun, it's not weird the way Erlang or LISP are, it definitely feels as Java 2.0 and it makes all kinds of Java deficiencies clearly visible. A few things look rather unnatural or necessary only for particular corner cases at first glance. But in general the language is definitely more sophisticated than Java. It also makes me believe that there is no reason to introduce major language features to Java anymore. It would feel as a different language and we would still have tons of legacy code which often does not use even generics. I did not think so even a year ago but I guess now I get what people were talking about for the last couple of years :)

There is no reason to cry about throwing closures out of Java7. Not everyone migrated even to Java6, not to mention those cursed with J2EE stuff chained to jdk1.4. It would be better to officially introduce, say, Scala as the next main JVM language and let Java peacefully lose its mind share. My guess is that we have learned enough from Java to not only wish for but also have a good idea of how a major overhaul of the JVM world should look like.

ctime: change or creation?

2009-01-05T16:35:00.000-08:00

Trying to understand how Apache runtime manages to deal with three different timestamps I stumbled upon a fascinating discussion of the same question in the context of underlying Unix file system. It turns out to be even more confusing than I thought.

Networking frameworks revisited

2009-01-02T11:25:00.000-08:00

Talking to a former colleague the other day I learned about recent changes in the world of NIO frameworks. The guy who led the MINA project joined JBoss and designed a new framework known as Netty. Previously, JBoss Remoting 3 was supposed to be based on MINA2. Now the original MINA and Netty designer is on Remoting team as well so I feel I cannot make sense of all this proliferation anymore (even not taking into account Grizzly). It remains to be seen if some grand unification/survival of the fittest will ensue.

While looking at JBoss documentation I also found that they have a new NIO-related project. It makes me wonder how dead Java is if a single company can produce so much new stuff in a pretty obscure niche.

Rightsizing

2008-12-15T12:26:00.000-08:00

So in exchange for an internal round from our original VCs we have cut headcount from 20 to 16 effectively now. Two from QA, one from support and another from sales. To me it is surprising that we kept all four developers because this is the most redundant area taking into account that our CTO does a lot of coding as well.

The new money is expected to take us through 2009. Our stated goal is to become cash flow-positive by the end of it. We are quite frugal and so we will probably be able to save on office lease and server hosting - the good news about the current environment is that prices are falling.

The good news is that despite the very same adverse environment we are actually seeing an increasing number of high-caliber customers. We have a business model and a limited but growing revenue. So I guess I should be cautiously optimistic about my employment next year even if I do not expect much from my stock options.

Respite

2008-12-12T11:25:00.000-08:00

Well, we seem to have secured another round and so my position is supposed to survive this time. Well, at least I have an up-to-date resume :)

Tell tale signs

2008-12-10T13:27:00.000-08:00

A Director of Engineering wrote a couple of LinkedIn recommendations. A colleague started job hunting. It bodes ill I guess. And I still cannot believe I should seriously consider the possibility of losing my job any day now. Probably because it would be the first lay-off in my career. I guess I'd better go update my resume.

It's coming

2008-12-09T14:22:00.000-08:00

Well, you know things are getting tense when your Director of Engineering questions his future with the company. Actually, we are very close to a new stage and all three options (another round with smaller valuation and headcount; being bought and left operating independently; being bought and merged with the buyer's organization) are still open this week. And it is very likely we will know the answer this week or so.

To be honest, I am getting moderately worried. I was hoping to keep this job until at least next spring. Let's see what happens, there are way too many uncertainties to invalidate any reasonable planning.

Recession repercussions

2008-12-05T17:20:00.000-08:00

So the crisis seems to be striking closer to home. During a weekly all-hands meeting today our CEO essentially predicted that by the end of the year we would either get a 4th round or be bought by somebody. And according to him although we have options, there is no way to predict how it all will end up.

Curiously enough, so far everyone I know have kept their jobs. Even quite a few former Lehman folks in NYC and London are nowadays employed by other well known financial institutions. Judging from increasingly negative news the next year is likely to be less merciful.

Which makes me wonder if it is safer to be in the Bay Area or somewhere where not everyone is a software developer. And speaking of job security, I am really curious whether employers will be slashing H1 people before residents and citizens or vice versa.

As a side-note, another former colleague just got his I-485 approved after only 4 months. Time flies - who could think a couple of years ago that after getting a GC the next challenge would be to stay employed :)

FP and LISP for the rest of us

2008-11-24T18:09:00.000-08:00

As a developer with a traditional C++/Java background and a non-CS major I find the recent surge in popularity of functional programming pretty confusing. It turns out that there is a whole parallel universe of FP where most hard-won imperative experience is of little help in understanding new concepts. And those are frequently hidden behind bizarre syntax.

I have stumbled upon a couple of very well written articles on FP and LISP . The author starts from something as trivial as Java and Ant and ends up explaining, say, the power of LISP. Very approachable with quite a few insights.

Real estate update

2008-11-23T22:37:00.000-08:00

Last weekend I visited a former colleague in her new house in Sunnyvale. Well, the house is a typical for the area 40 or so years old structure with a pitiful backyard of about 5,000 sq ft. And it faces a 4-lane road. In a rather mediocre school district. They bought it for 750K in late spring.

Now do not get me wrong, I know that this insane price tag is somehow considered normal in the SFBA. And it is convenient to commute from there to most of the peninsula. And they had no choice taking into account their tax bracket - both of them are very smart people working in software. But boy, how ridiculous this price is.

And now for the really funny part which beautifully illustrates the regretful state of affairs in this country. Before the house was foreclosed it had been occupied by Mexicans. For 900K. Yep, you heard me right - some moronic American banker gave Mexicans 900K in Silicon Valley, I am not kidding you. And neither the banker nor the peasants will suffer, only normal people with down payments and common sense. Only in America ..

I tell you what - people I know do not dare to take such a mortgage even with family incomes close to 300K. And some people still wonder why prices are so crazy here.

Date and time in Java

2008-11-06T15:57:00.000-08:00

Today I was fixing a bug which revealed a problem in how we incremented dates (yes, by adding 86400000). Naturally, it led to corrupted data after recent switch back to Pacific Standard Time from Pacific Daylight Time. I had to go review a very good overview of time-related Java issues and play with a date calculator.

Frankly, it's still rather counter-intuitive to me how much complexity is behind seemingly innocent long timestamps. What could be simpler than milliseconds passed since 1970? ;)

Event Stream Processing

2008-11-02T19:16:00.000-08:00

I have just stumbled upon this new approach to event-based design. A wikipedia example says it all in 7 lines. The idea is to use an SQL-like language to create filters (capable of event correlation, typically limited to a certain period) and run a continuous stream of events through them. There is even an open source implementation. At first glance most currently available products are used in algorithmic trading (whatever was left of it after the crisis :) ).

To me this concept seems to be a kind of rule engine and so potentially there are multiple domains where it might be applied. On the other hand it makes me wonder whether it is indeed a re-packaged rule engine and in such a case what is so different about it.

Performance is the name of the game because ESP is positioned as a soft real-time technology. It must be pretty interesting to read how they manage to achieve their declared ability to process a huge number of events per second on commodity hardware.

Bay Area Scala Enthusiasts

2008-10-15T13:19:00.000-07:00

There are a few user group-like communities I am aware of

Unfortunately they frequently meet somewhere on the peninsula which is a decisive no-go for people working in SF and commuting on BART.

Luckily, the latest Bay Area Scala Enthusiasts meeting was held at Twitter HQ in downtown SF yesterday. Naturally, I could not resist the temptation of walking 0.8 mile. The people were very nice and it felt so good to be in a true geek crowd. It would be great to attend such gatherings on a permanent basis so I am crossing my fingers hoping that the organizers will pay more attention to San Francisco.

The official presentation was a rather lame discussion of a nascent coding guideline. I would say that this is a non-issue (at least in the scope they mentioned) usually addressed by each team (e.g. questions such as "2-space v 4-space v tab formatting"). The fact that some people come to Scala from fairly uncommon scripting backgrounds complicated the discussion. Later the conversation shifted to real life experiences and favorite tools and that part was more instructive.

It was mentioned that in contrast to Java the question of Scala style is open because different approaches are used in Scala standard library. Many people are disappointed to discover that behind the functional API there is a lot of imperative code. In other words, although Scala seems to be the front runner in Java 2.0 race there is very little infrastructure (both conceptual - such as books/idioms/style guidelines and concrete - such as DI containers) in place or even under active development. And it looks like there is still enough confusion whether to re-use/wrap Java components or re-implement them from scratch.

Actually, I am surprised to find out that non-Java folks are interested in Scala. Judging from blogs web-heads are into dynamic /scripting languages such as Ruby. I believe Scala should be more appealing to people who moved to Java from C++ or find Erlang too weird because of its OOP-unfriendly nature.

Improved Scala support in Idea

2008-09-14T13:30:00.000-07:00

The other day I noticed that IntelliJ had issued an updated Scala plugin. Unfortunately it works in Idea 8 only (with a good justification) so I had to install its beta as well. The good news is that it was worth it. I remember the original plugin which did not qualify even as a toy. This time they made real progress.

On the positive side now there is decent syntax highlighting, Ctrl-click navigation and other basic Idea facilities actually working. Clearly there is a lot of space for improvement from better code correctness checks to Scala javadoc support in Ctrl-Q but at least now it is possible to play with the language in a meaningful way.

This year I have seen so many blogs telling about Java decline and the rise of dynamic and/or functional languages that I decided it's time to extend my horizons. I decided to look at Scala (as a "better C++ than Java") and Erlang (because of its telco roots and the availability of a very good book). So far I have been struggling mightily with weird syntax of functional languages.

What I really find puzzling is that despite all the buzz about superiority of dynamic/functional languages I have not seen a single book on design with them. The standard OOA&D cannot be completely applicable (e.g. there are no classes in Erlang and no interfaces in a typical dynamic language) and even UML is likely to be more convenient for pure OO. Taking into account how long and hard it was for most people to master true OOP I am surprised with the lack of such discussions.

Development documentation and Wiki

2008-09-02T10:40:00.000-07:00

For me Wiki is a relatively new development documentation media in comparison with Word documents. Although I understand its advantages I am also familiar with its inconveniences. To me Wiki seems to be a compromise where we essentially trade order for universal accessibility.

Recently I had an opportunity to compare the two approaches. Having implemented a new component I wanted to describe it for posterity at the end of the sprint. For simplicity sake I drafted a Word version of the design specification on the basis of a template I came up with a few years ago.

It should be noted that our startup would hardly win any award for the quality of development documentation. In our line of business even established companies struggle with it and fashionable Agile methodologies provided well-intentioned justifications for getting rid of it altogether. So my second goal was to use the opportunity to give a good example of how decent documentation looks like.

I was not surprised to know that although our Director of Engineering appreciated the content he immediately asked me to convert it into Wiki pages. I should admit not all of his objections sound convincing to me. I would even argue many of them are nothing more than a lame excuse for developers who just do not care. Among them were:

It is hard to update, especially in a collaborative fashion. This means that it might easily become out of date as the system changes.
It is hard to index and reference. A multi-page wiki document can be easily bookmarked, watched for changes, and referenced from bugs/tasks in the future.
There is a level of formalism (naming and numbering conventions) that might discourage contributions from people other than the original author.

Just to be balanced here is my Wiki hate list:

It's impossible to have a document template (analogous to ".dot" files in Word)
It's impossible to baseline a document together with code in VCS
It's impossible to version a document as a whole
Splitting a long document into Wiki pages is painful (you need a page naming convention, it's significantly less convenient to format, there is no support for automatic section numbering at different levels, there is no way to generate a TOC)
It's difficult to print the whole document

In other words, Wiki loses a lot of power in comparison with real documents in exchange for, basically, WWW-like look and feel. From a more practical perspective, there are a few question to answer before you write a Wiki document or migrate from a Word one.

A dedicated wiki section. You will need a root for the hierarchy of development documents. Virtually all the wikis I have seen were structured by departments at a high level (e.g. Development, Operations, QA). Under Development pages are typically grouped by type rather than subsystem as well. So although theoretically there might be already a sub-tree for each component in your system I would expect to find or start a new tree somewhere under Development (e.g. Development/Development documentation/Component X)
A standardized tree structure for every component. A conceptual document will need to be split into multiple physical Wiki pages if only to keep them short enough. It' is important to keep a uniform tree structure for all the components.
A page naming convention. There are a few different page types such as a component root (e.g. "Component X"), a document root (e.g. "Component X/Design specification") and document chapter (e.g. "Component X/Design specification/Static view"). It would be even messier if a chapter were comprised of multiple pages. Now those were just page titles, real page names would be like "componentx", "componentx_sds" and "componentx_sds_static".
To have access to multiple versions of the same document pages should have versions. Although Wiki keeps track of page changes it is of little convenience if you want just to read a particular final version (as opposed to searching through multiple drafts with highlighted changes). So I would expect all the pages to carry component version numbers as well. Consequently, page names are likely to resemble "componentx_1_0_0", "componentx_1_0_0_sds" and "componentx_1_0_0_sds_static".

Pair programming in real life

2008-08-12T17:18:00.000-07:00

I am not much of an Agile fan. With a degree in engineering and a background in telco software I came to appreciate the architecture-centered mindset, development methodologies and documentation. Nevertheless, the Agile movement has been very important for marketing of multiple best practices which we all know and love. Just think of TDD and how development had felt before xUnit came to our rescue.

Arguably, XP is the most controversial approach and Pair Programming the most contentious practice it introduced. Personally, I never felt good about it. There are very few people I would tolerate that close and I do not like people touching my keyboard and mouse for sanitary reasons anyway.

There seems to be a more benign variant of this practice though. I have noticed it in more than one company so it seems to be quite common. Reading a good overview of Agile practices recently I found that there is a name for it in FDD: a feature team. The idea is that a feature or component is assigned to a small team, not a single developer. In my experience it usually takes a 2-developer team to be really productive so the parallel with Pair Programming is self-evident.

The benefit is clear. You can bounce ideas of each other, you are likely to have different areas of expertise or at least skill levels, you can complete the assignment almost twice as fast and there will be two people knowledgeable of that particular component.

Promoting Maven2

2008-08-06T16:43:00.000-07:00

As a relatively small company we have a lot of rather messy code to maintain and not much spare capacity to significantly refactor it immediately. All kinds of bad smells are present - from having just two huge binaries shared by drastically different components to having compiled unit tests in those two production JAR files. It makes me cringe every time I think about.

Curiously enough, once upon a time there was a pretty good (at least according to the founder who is a competent software developer) reason for putting everything into one binary. It was easy to updated multiple servers and keeps things simple in general, at least from the operational perspective.

For more than three months I have been pushing for better software engineering practices in a few different areas. My first win was persuading our Director of Engineering to allow me to use Maven2 as the build tool for a new component we designed in this last sprint. To me M2 feels pretty much the way the Spring Framework does - once you try it you cannot imagine you lived without it.

On my current team I turned out to be the only one with previous M2 experience so I expected troubles. From what I observed, people with no previous knowledge of dependency management systems tend to be less than excited about keeping their dependencies explicit. In a sense, it's like TDD - it takes certain changes in your mindset to realize how valuable it can be.

The good news is that another engineer we teamed up in the last sprint was easily convinced once he saw how easy it was to add new classes and unit tests without messing up with Ant-style classpathes/directories/proprietary targets. Luckily, our Bamboo CI server integrates with M2 as well so barring minor integration difficulties (such as the need to install a few proprietary libraries built with the old Ant-based approach into the M2 repository) we are all set.

I really like M2. It's simple but flexible. It has tons of plugins - it took me just ten XML lines or so to add Cobertura to out build. And the really good news is that finally we have two extremely good free sources of documentation. I remember struggling mightily with M2 a couple of years ago when their infamous site was pretty much all there was. Nowadays, you can just go and download either a more introductory book or a more reference-like volume. Even search in public repositories hardly could be made easier.

Reporting EOW

2008-08-01T13:03:00.000-07:00

As some might know, CA is ruled by socialists. It comes as a surprise to people coming to the US because from abroad America looks as close to a paradise for hard-working professionals as it gets. And as most of us grew up thinking of Silicon Valley as the center of the Universe and the US in particular we are in for a rather nasty surprise. Taking into account the demographics, it does not sound strange to me that most Americans do not actually consider this state a part of the country anymore.

If not for the weather (the only state with Mediterranean climate) and natural beauty (three or four hours of driving to such stunning places as Yosemite or Lake Tahoe) I would be inclined to consider moving to a more hospitable place. And I fully expect I will - Dems don't even hide the fact that they are going to raise taxes and extend welfare to the lazy, the stupid and the illegal. If they are as dangerous before the elections one can imagine to what an extend they will be able to ruin this country if they, God forbid, actually win. Guess who is going to pay for it?

The other day I was looking at RE prices in Plano, TX - a seemingly nice suburb with relatively good demographics. You won't believe what $700K can buy you there - marvelous 4000+ ft houses built in 2008-2006 (sic! ever heard about anything built after 1970 in decent parts of the Bay Area? and for how many millions exactly?). Again, if not for average night temperatures of 70+ in summer I would be heading there right now.

So my point I guess is: don't come to CA dreaming of the Valley. Consider TX, NC, WA (and hey, in many of them there is no state tax at all!). Probably CO if you can tolerate snow. CA is not a place where a normal person would like to live and eventually raise children.

On the brighter side, I ran across a hilarious succinct way to describe the prevailing local mood:

How various newspapers would report the imminent end of the world:

The Wall Street Journal: "END OF WORLD FUELS BOND MARKET JITTERS".

USA Today: "END OF WORLD - HOW WE FEEL ABOUT IT - POLLS, CHARTS, GRAPHS ON PAGE A4".

SF Chronicle: "WORLD TO END TOMORROW - WOMEN, MINORITIES HARDEST HIT".

JAOO 2007

2008-07-13T13:31:00.000-07:00

JAOO Video on blip.tv

Some things never change

2008-06-25T10:28:00.000-07:00

This hilarious email from Bill Gates contains an instructive story. Founders are obsessed with their companies and quality of their products. But when a team grows beyond its original core all sorts of people come in. They typically concentrate on their parts of the system and pay less attention to the whole thing. Or simply expect somebody else to take care of it.

I definitely saw startup CEOs/Founders frustrated by short-sightedness of their employees. A founder is expected to be a visionary as well and this could explain why more tactically oriented people miss important points. Seeing it at the scale of MSFT indicates that it's indeed an organizational pattern.

Still, the email is absolutely hysterical :)

New NIO

2008-06-15T23:15:00.000-07:00

A GOOG talk based on JavaOne presentation on NIO updates coming in Java7.

This [beep] networking

2008-05-30T13:47:00.000-07:00

It turns out there is a network protocol framework introduced a few years ago and covered by a couple of RFC standards. I have not looked at the design or source code yet. Taking into account all the open source middleware I know does not use it I am wondering whether anything at all is reused in the middleware transport layer.

high-scale-lib presentation

2008-05-16T09:59:00.000-07:00

I mentioned before this new library of nonblocking data structures. Its author prepared a very interesting presentation explaining some of the internals.

Terracotta at JavaOne

2008-05-08T16:38:00.000-07:00

Today I went to see what was going on at JavaOne which is held a couple of blocks from my office. The Pavilion was not much different from last year so I ended up talking mostly to Terracotta guys and brainy Taylor in particular. I was asking different nasty questions about their approach and he was kind enough to talk to me for at least half an hour. Right after he talked to Brian Goetz himself.

My opening salvo quite naturally was about their messaging framework. I was essentially told that JGroups clusters of more than 4 servers had been widely known and Apache Tribes did not fit exactly and bug-fixing was insufficiently quick. From what I know the whole JBoss stack (at least in its JBoss AS 5 reincarnation) is supposed to depend on JGroups and so I am at a loss to reconcile such contradictory statements. Probably the key is the number of servers in a cluster.

At face value it means there is no open source reliable messaging framework capable of scalability beyond a couple of servers. Taking into account that pretty much anything which is not yet "in the cloud" is clustered nowadays that sounds odd. And it drives home the idea of reliable messaging as a truly challenging thing to make of production quality.

I learned more about there positioning as well. They are after the middle market of, roughly speaking, up to 50 servers in domains such as web application. Which I guess implies that JBoss is a competitor more than Coherence/Gigaspaces going after larger clusters in Finance.

An active-active L2 server configuration is expected by the end of the year although the common belief is that the 10 seconds required to switch to a backup server now are tolerable. From how I understood it they are planning to send separate updates to both L2 servers instead of multicast or replication between the two. I might have misconstrued something though.

We talked about their paradigm a little bit. I admit to being rather uncomfortable with it because they are the only company I know literally exploiting the conceptual similarity of concurrent and distributed systems (i.e. CPUs sitting on the same bus differ from servers in a cluster only by communication delays much more pronounced in the case of a LAN). It is so different from pretty much any product (exposing a real API in terms of actual interfaces in, say, JCache as opposed to delimiting transactions with monitorenter/monitorexit pairs) that either they have invented the best thing since sliced bread or they are likely to fail as mavericks. They might as well be the next "the network is the computer" after all.

The foundational paradigm of Terracotta as a distributed JVM (complete with a DGC) evokes the same kind of argument as JVM used to ten years ago. Back then the idea of Java performance comparable with C++ was ridiculous although it was said at the very beginning that JIT-style dynamic optimizations would do the trick one day. It looks like JVM guys have pulled the trick after all so this lesson may have significant implications for Terracotta.

As an example, it can detect that a particular instance is used exclusively by one L1 server and transfer lock ownership to that L1 server from the central L2 host and so effectively avoid using distributed locking. As a result we have a sort of buddy replication (between L1 and L2). Like a JVM silently eliminating synchronization in a sequential program. Theoretically neat :)

One thing I can say safely is that in contrast to many companies in this field Terracotta is not afraid to share its source code. They do not pretend like Coherences of this world do that someone can steal anything from the code base and ruin their empire (anybody heard of a new M$ after the windows source code was leaked on the net?). As a developer I believe that code quality says a lot about the corresponding system (not to mention things one can learn from a large successful system) and I applaud Terracotta for their bravery.

NIH and reliable messaging

2008-04-23T16:15:00.000-07:00

We are planning to use Terracotta as our distributed caching infrastructure. I always thought this framework was rather odd - pretty much every one else follows the usual "JGroups-like" approach of providing an API manipulating with such abstractions as Channel/Group/Member or the recent Map/Reduce-inspired analogues.

For historical reasons I am curious about different transport-level frameworks. Last year I was looking closely at JBoss and was amazed by their indiscriminate use of a few overlapping technologies such as JGroups and JBoss Remoting (which they were planning to migrate to MINA anyway). Coherence has its own TCMP protocol and so does Gigaspaces. And a new messaging standard will likely influence the field. I certainly understand why developers are so inclined to re-implement the same functionality. Although there are well-known patterns for designing such a component there is a lot of hard-core fun left in all the intricate details of harnessing NIO and multithreading.

The flip side is that this kind of software is extremely difficult to get right. Even years later there could be elusive bugs (I heard JGroups still fails when there are a few dozen severs) . From my experience hunting them even in a 4-server configuration can be nightmarish to put it mildly. So I guess for those who do it implementing such a framework is a major professional self-esteem booster.

Naturally, one of the first things I was looking at in the Terracotta code base was their clustering. Curiously enough, they borrowed their transport layer - I guess they were having enough fun with higher-level state clustering. But the layer itself proved my point again - it's the framework used in Tomcat 6. So ASF seems to be following the same route of developing multiple overlapping technologies as JBoss is.

Employee #25

2008-04-14T17:32:00.000-07:00

It was a hectic day of getting to know my new company. An engineering team of fewer than a dozen, a convenient location just across the street from a BART station and nice views of SF, decent benefits. I have not seen a single line of code yet but I know that my initial assignments will be about Apache modules (well, C is not as sexy as C++ but hey, any of those would be fun after so many years of Java) and Terracotta.

Pulling the trigger

2008-03-31T11:04:00.000-07:00

On Friday I sent a 2-week notice. I will be joining a startup in San Francisco in a couple of weeks. Time will tell how smart is was to choose a startup amid a recession but so far I am looking forward with genuine excitement.

Curiously enough, not sooner had I accepted their offer than interview invitations started piling up on my voice mail. There was even a telecom-related startup but strangely enough no biotech companies (I guess those are much more likely to be located in the valley, not in SF). Nevertheless it can easily take a month to find an interesting company so in this respect the SFBA must be not different from less famous places.

And I am still under the impression that only a minority of development positions in the Bay Area are truly exciting. Not all startups are created equal and not all straightforward enterprise Java jobs went to India yet. Taking into account a growing number of acquaintances working in really interesting scientific startups in such places as WA or DC it looks like the SFBA is more exceptional because of its weather than its technological prowess.

Consistent Hashing

2008-03-19T12:27:00.000-07:00

An interesting and conceptually simple approach to choosing what to put on a new cluster member or where to move the state of a failed one.

Wrapping up my job hunting

2008-03-19T08:47:00.000-07:00

I have finally got an official offer from a startup in SF. I have to make up my mind by Fri and I am very inclined to jump in although their development has nothing to do with telecom or anything like that. The only potential but serious problem might be the idea of joining a startup when the economy is plunging (to keep myself sane I avoid news but even the bits I hear occasionally sound rather ominous).

Well, I did not really come to the US in order to stay on the safe side even though I tend to enjoy my comfort zone :) I believe I am missing the excitement of working in a startup. Even if my stock options vest without bringing any cash this kind of experience is virtually guaranteed to be beneficial in so many respects. And I am getting old enough to recognize that following one's desires is arguably the most efficient way to have few regrets in the future.

On the positive side, their benefits are quite decent, they have an agile development process in place, they are about to face interesting scalability/availability challenges, there are more experienced people on the team. On the other hand, the system is not overly complex yet and who knows when they will truly entertain the idea of clustering, I would move even further from the domains I know and into the field of web systems and advertising.

Anyway, the feeling reminds me of the old good jnetx when both Tom and excitement were still present. Let's see what happens next.

If you do not succeed at first ..

2008-03-14T15:28:00.000-07:00

The second week of job hunting has confirmed my original observation - I have no idea why some companies like me and others do not. So far those looking for a typical enterprise mix of JEE/Hibernate/Spring/web UI have been mostly ignoring me (Am I positioning myself as too smart by mentioning multithreading and clustering? Do they think that less than 5 years of EJB2 or Struts or scripting is not enough?). At least in San Francisco most Java positions are about web-based development (and mostly of the boring JSP/AJAX/Struts kind rather than GOOG-like scalability/availability). It never ceases to puzzle me why they have not outsourced that kind of development yet.

I was also neglected by a handful of telecom-related companies in the valley. I am getting scared thinking that my recent couple of years of enterprise Java could be the reason. Telecom knowledge might be a commodity in the valley. Then again, at BEA they were really exciting about my experience. Anyway, there were probably just two ISVs and Cisco in this category.

So far I have had more success with startups (no high-tech, engineering or even bio-tech, just one more social network, ad-serving service or media-related stuff). I used to think that they are the most finicky employers and I was surprised to get a few interviews with them. They did not like me in one place, I am likely to reject another company because of their size/technology/business direction and I will be meeting the founder of my current favorite next week in a final round.

I have virtually lost any hope to get a telecom job (there are only few of them out there). Despite all the media hype on multicore CPUs and the coming Age of Concurrency I do not really see even util.concurrent among requirements. The same goes for grids - just a couple of senior positions (and a few more in old good C/C++), no idea how to land one without significant relevant experience.

The good news is that startups indeed pay as much as other companies and tend to have comparable benefits (insurances, weeks of vacation, 401K) so the stories of sleeping under work desks for a pile of stock options seem to be a bit out of touch with reality.

Interview-quality concurrent primitives

2008-03-10T12:10:00.000-07:00

The exciting thing about interviews is that you hardly ever know how any of those will be conducted. Take concurrency as an example. It could be ignored, some people may ask you about plain vanilla wait/notify semantics, other about using primitives from the util.concurrent library, and then someone is likely to ask you about arguably more intellectual algorithmic side. In the latter case, a typical task is to implement something like a thread-safe bounded buffer or a read-write lock.

Well, I guess except for Messrs Lea and Goetz few would probably attempt to roll out a production quality CAS-based design having just a white board and half an hour. Which means that knowing simpler approaches makes sense at least in the context of interviews. Here where a simplistic book can help.

A half of it is dedicated to the RTSJ which seems to be more of a research toy. The other half provides a rather messy introduction to concurrency. I used to consider this book to be mostly useless (too thin theoretically, insufficiently util.concurrent-oriented for real-life development). But a few basic implementations of primitives such as semaphores and locks certainly can be used for interview purposes.

Java jobs in San Francisco

2008-03-07T15:45:00.000-08:00

Well, I am not sure whether it's the recession or something else but the positions available on the market are mostly lame IT jobs in enterprise Java (JEE/Spring/Hibernate and quite often even offering server-side developers to mess up with web UI if not some obscure scripting languages). Telco is almost unheard of, just a couple of jobs in Finance. Another observation is that even those few startups located in SF seem to be working on some weird stuff like social networking or mobile phone-related gaming/ads/photo sharing.

It looks like there is a significant difference between SF and the Valley despite just a few miles between them. In my opinion there are noticeably different positions in the Valley (including telco, multithreading and other interesting things having at least some resemblance to engineering). I am looking forward to moving to Fremont (because of family circumstances) in a couple of months. From there I will be able to drive to the Valley and hopefully find something good enough to uphold the image of this area as the software capital of the world.

The problem is that the Valley is completely cut off from the world and going there means not only buying a car but also actually driving it daily instead of some productive time spent in BART reading or even working (not to mention even more expensive accommodations or the fact that my wife won't be able to go anywhere on her own).

The most annoying thing so far has been recruiters (yes, surprisingly enough they exist here in the US as well). I am totally sick and tired of their seemingly incurable desire to talk on the phone, instead of just forwarding my resume to an actual decision maker. Many of them are stupid enough to call even before they actually send me a job description. Probably this is how stories about "stupid Americans" originate.

PS
A couple of nice questions I was asked today:

There is a table with two columns (ID and Name), the same Name can be used an arbitrary number of times with different IDs. Write a SQL statement removing name copies so that for each Name only a record with the lowest ID value is left
Implement a ReadWriteLock (hint: beware of livelock and fairness)

A better alternative to Ruby?

2008-03-05T11:11:00.000-08:00

With all the noise about dying Java and shiny new languages coming it might be a good strategy to look at those. Who knows, if Java managed to overcome a vastly more sophisticated language something even less appealing might take over Java. The bad news is that the number of those new toy languages seems as ridiculous as their usually atrocious syntax.

Bjarne seems to be the last thoughtful language designer out there and script kiddies seem to prefer reinventing wheels. The latest case in point - in addition to a lame thread implementation, Ruby repeated the very same mistakes Java fixed years ago (I thought so much better about Japanese engineers). It's all well and good to add new concepts but why on Earth would one want to introduce his own syntax for things which have been polished for years in successful languages? NIH?

My most frightening hypothesis is that Java was cursed from the beginning to be associated with web development. The majority of Java positions I see even in San Francisco are about churning out JEE or at best Spring/Hibernate code. In my opinion this is IT, not software development. Naturally, there must be easier ways to do web UI than Java and for that niche Ruby&Co could be a viable next generation. The problem is, I do not want to be a web developer, I want to be a software engineer equipped with a sophisticated general purpose language, not a dumbed-down script designed for village dwellers from developing countries.

Another interesting aspect is that virtually all the hype is about languages which are likely to be VM-based. I hardly can imagine anybody beating years of major VMs optimization by moonlighting so any new serious interpreted language is likely to be implemented on the basis of JVM or CRL. Even here Ruby managed to go its own way, at least up until recently.

I have just stumbled upon an article (admittedly, its ludicrous "java as a first love" introduction may scare away) on another contender. I heard the word Scala before but never cared enough to look at it. And in contrast to 15 or so book on Ruby there seems to be the only one on Scala. Well, at first glance Scala is an interesting option to look at, even if only as an educational resource for functional programming. It's compiled into Java byte code, looks relatively more normal (but it uses square brackets for parameterized types .. what is wrong with those new language designers?) and seems to be a good fit for multithreading.

Distributed algorithms

2008-03-03T14:18:00.000-08:00

Distributed algorithms are a fascinating but often overlooked area in which academic research meets real-life challenges. Well-known practical applications such as the JGroups framework tend to simplify (for performance reasons obviously, certain levels of consistency are prohibitively expensive in terms of network traffic or latency) the approaches suggested by academicians. On the other hand, virtually all such frameworks are firmly rooted in hard-core research (e.g. Bela Ban was influenced by research done in Cornell where Ken Birman seems to be the founding guru) and you won't go far on common sense and re-invented wheels.

In mainstream development layers upon layers of middleware protect programmers from the challenges of real engineering. Quite few are lucky enough to closely work with JGroups-level components and even fewer use NIO and MINA-like frameworks (BTW MINA's lead developer has just been hired by JBoss) to build infrastructure of their own. In my experience, even among CS graduates not everyone has a good grasp of this field. Once upon a time I struggled mightily to learn basics of reliable messaging and its internals. As a result, I have been keeping my eye on the literature but surprisingly enough for our age of grid computing there are only a few sources catering to practicing engineers. Most CS books tend to jump to the theoretical aspects and never come back to something more practical.

So far I can truly recommend only a couple of books paying attention to real code.

Concurrent and distributed computing in Java - a very good introductory book covering both essential concurrency issues and distributed algorithms. The author did a very good job of actually explaining many concepts and their simple Java implementations (the code is probably not perfect but it makes so much easier to grasp the ideas) without overwhelming the reader with gratuitous mathematical rigor. The book consists of chapters of roughly equal size which makes it possible to read in manageable chunks. Each chapter refers to selected original articles and so explains the history of the field, again very succinctly.

Introduction to Reliable Distributed Programming - this book is based on a real-world messaging framework and as such provides both an in-depth discussion of algorithms and their implementation. The only thing I can say is that I wish there were a comparable book on JGroups which is arguably the leading Java framework.

If I live long enough to overgrow those two books I will probably read also Design and analysis of distributed algorithms but I am not entirely convinced it will happen before I retire :)

As a side-note, I found the JGroups source code rather messy. I frequently saw places where refactoring was long due. Consequently, I doubt that framework is a good educational instrument at least for low-level mechanics especially if you do not have a pretty good idea of what exactly to expect. My UML sketch of TCP/UDP stacks looks rather unwieldy but they say most highly useful pieces of software look puzzling inside.

A first look at Ruby

2008-02-21T15:41:00.000-08:00

In preparation for job hunting later this year I have been brushing up on a wide range of topics. Probably the least exciting one of those is web services. I wholeheartedly hate what is known as web development (e.g. JSP, Web MVC frameworks and web services themselves) and I hope I will be able to find a job as remote from those as possible. Nevertheless I am reading a couple of books on traditional WS-xxx and RESTful approaches to keep myself aware of web-related architectures.

For better or worse, the second book is illustrated with examples in Ruby. This language has been discussed a lot recently (mostly in the context of its web-development framework) by pretty smart people including Martin Fowler himself. Some even argued that Ruby will be the next big thing. Although any mentioning of web development or scripting raises a red flag for me but this time I decided to invested a couple of hours into learning the basic syntax of Ruby just to get through the aforementioned book.

Boy, what a mess Ruby is at first glance. I grew up believing C++ was the golden standard of programming languages (probably because I was too young to start from Lisp or Smalltalk) and I think Java is a hopelessly watered-down leaf of that language branch. Now Ruby makes Java look like a proper language. I do recognize that there are certain smart concepts implemented in Ruby (e.g. closures and - hurray! - operator overloading and default parameters sorely missing in Java) but why on Earth have they chosen such horrific syntax? Pascal-like block demarcation with "def" and "end" instead of normal brackets and constructor names different from the class name? The naming convention based on lower case and underscores? Why could not they borrow proper constructions from C++? Those little details ruin the whole experience (and long before we start asking questions about serious stuff such as concurrency support).

Another funny fact is that searching for "Ruby for Java programmers" retrieves a few articles right on the first page while "Ruby for C++ programmers" does not. There must be something about this juxtaposition :)

As for the first book on WS-xxx .. the book is pretty good and properly covers the way to do them in JEE5. But in my opinion all the gory details are the most cogent evidence that WS are just a lame and bloated reincarnation of CORBA in a gratuitously verbose format. That RESTful approach could be to WS what the Spring framework was to EJB2.

Vacationing in Southern California

2008-02-15T09:32:00.000-08:00

I am back from my first real American vacation - 1,456 miles in 6 days. I am still slightly dizzy after driving 675 miles from 7.30 AM to 8 PM yesterday. But it was worth it, every single mile :) A night in LA, two in San Diego, two in Las Vegas and the whole morning of driving through Death Valley. And it was unusually warm (about 80+ instead of average 70 something) for the season most of the time.

LA turned out to be grossly overestimated. Aside from a handful of things deserving attention it's not a place to spend more than a night at.

Griffith observatory - for the impressive view of the city from it, very nice scientific artifacts and proximity to the Hollywood sign
Hollywood boulevard - well, one is supposed to see those stars on the sidewalk but the street itself won't stun you even at night
Beverly Hills - mostly so that you (or more likely your wife) can tell your friends you have been there but also because the district looks impressive to such an extent that it's probably the only place in CA where real estate prices are justified
Mulholland Drive - for a few more impressive views of the city

In sharp contrast to LA I found San Diego to be charming. It's tidy and quite walkable so it was the first American city I wished I had lived in. My list of things to do would include:

Walking down Harbor Dr in downtown to Marina Park
A very good aircraft carrier museum - the one in NYC is likely to be the only real competitor
Balboa Park - absolutely stunningly beautiful, palms and museums in European-style buildings. The park alone makes me want to live in San Diego.
Driving on the Coronado Bridge at night for panoramic city views
Old Town San Diego - a typical American "restored old town"
Gaslamp Quarter - nothing really special but you will likely dine there anyway

I expected Las Vegas to be overly artificial and it was. Nevertheless it was really fun at night with all those neon lights and quite visually appealing in daylight. So although it has kitch written all over I think anyone will enjoy a couple of day in Las Vegas. The only huge problem is that it is allowed to smoke, even inside of casinos.

Death Valley was an unexpected gem on my way back. I half-expected it to be sort of a desert but it was so much more. This mountainous region (you see signs such as "elevation 3000 ft" more than once) is so picturesque you have to see it to believe. In addition, CA 190 going through the park is in perfect condition so driving there is a pleasure in itself.

The whole adventure cost me $805 ($418 for five nights in motels, $231 for renting a car for six days and $156 for gas). I found the car to be quite nice and as noisy on our imperfect CA highways as the last Corolla I drove. Highway MPG averaged at decent 35 which I did not really expect from a Chevy.

util.concurrent Synchronizer Framework

2008-02-06T08:45:00.000-08:00

There is a lot of stuff on Doug's page so it is relatively easy to miss something. I did and only by chance came across of this paper. In 6 terse pages (a few more are dedicated to performance measurements) it covers the design of the AbstractQueuedSynchronizer class on which many implementations depend in the framework. Highly recommended.

non-blocking util.concurrent

2008-02-05T14:16:00.000-08:00

I used to think that Doug Lea was the only Java concurrency god out there and that his library would stay the ultimate achievement. I know that the second Java concurrency bible was a joint venture but even after reading it cover-to-cover four times I see it as a significantly updated revision of the original Doug's bible. After all he authored the library, others mostly extended the java docs :)

I stumbled upon a small non-blocking contender. It's limited but interesting because it promotes the non-blocking (i.e. CAS-based) approach. The developer's credentials are stunning although the code itself looks like a mongrel between Java and C. Anyway let's see, we might be witnessing a nascent branch of util.concurrent in Java 7+

A Trojan War

2008-01-03T10:03:00.000-08:00

A quiet evening turned yesterday into a prolonged battle against a Trojan virus. Apparently that nasty piece of software was only mildly malicious but it held well against my initial curing attempts. In other words, I managed to go to bed slightly after 3AM :)

Having moved from manly C++ to Java and the aloof world of VMs years ago I have been occasionally struggling with Windows specifics. Naturally, I keep vague memories of Win32 and DLL Hell but I am not a frequent user of regedit so to speak. So when my anti virus scanner detected the intruder but failed to clean or remove the infected DLL I grew rather uneasy. After a while I recognized it would make sense to go and see what the virus was and how other people subdued it.

Major vendors provided detailed recommendations on the Trojan and the registry entries typically affected. There was even a dedicated tool from Symantec. The suggested approach included rebooting in safe mode, running Winternals Process Explorer to suspend a few processes such as Explorer and Winlogon and letting the anti virus software do its job. Which was all well and good until it became abundantly clear that the infected DLL was loaded by those suspended processes and as a result could not be removed anyway.

Things were definitely getting interesting after midnight. I did not exactly let my IQ shine and only after reading online comments from other victims I thought about booting from a different source. I used the majestic Winternals ERD Commander and effortlessly removed the offending DLL. The bad news was that rebooting resulted in a missing Explorer process (no task bar or Start menu but surprisingly the Task Manager still worked and allowed to launch applications).

The good news was that I had hints concerning the affected registry entries. The age-old game of dependency-hunting led me from GUIDs used by Explorer and Winlogon processes to the registry entries of the infected DLL. Once I found and erased them I could reboot and see the processes successfully resolve their dependencies. The night was invigorating but I will have to be high on caffeine today.

Christmas sales

2007-12-28T16:18:00.000-08:00

On Wednesday I went shopping for a second external hard drive. I was going to buy a 500GB drive for about $150. A holiday discount came as a nice surprise and resulted in a 750GB drive for the same price.

Many years ago I was told that no matter what the capacity of one's HDD was the disk would be 80% full in half a year. This rule of thumb has held surprisingly true for years (even for my first 300GB backup drive) but my imagine fails to come up with a way to repeat this feat with the new seemingly bottomless toy.

Weekend trips in the Bay Area

2007-12-13T11:53:00.000-08:00

Two of the local advantages equally envied by Americans from less fortunate states and many foreigners are the climate and the abundance of natural beauty within day- or weekend-long reach. The former results in so many sunny days that clouds are more of a curiosity and a weather-inspired depression cannot be imagined. Yes, nights are rather cold in winter but the blue sky and the shining sun in the morning easily offset this mild inconvenience. At first glance temperatures are comparable in NC, TX or FL but the consensus is that here the humidity is much better.

Probably the only thing I miss in the US is historical sites (e.g. Medieval towns and art museums so abundant in the EU). Americans compensate themselves for this lack of historical artifacts with natural marvels. Parks, from tiny municipal to world-renown national are everywhere. Seashores, redwood forests, mountains. Nicely maintained and easily accessible. No wonder most people prefer traveling domestically or within the continent at best.

[G]reat [C]elebration

2007-11-15T11:00:00.000-08:00

Well, I still cannot believe my eyes. The adrenaline rush has mostly ebbed but not entirely :) I was easily overwhelmed with the news mostly because I did not get a notification on I-140 approval so I had no reason to expect the end of my travail so soon. Until I have got my hands on the card itself I consider it unofficial though :) And I wonder how longer it will take for my wife's application to be approved.

For those of you interested in statistics:

Arrived to the US in late Jan
Got LC approved on Mar 19
Sent our I-485/I-140/EAD applications in first week of May
Fingerprints were taken at the end of May
EAD received in late Jun
RFE on the ability to pay (I-140) was received on Aug 28
The RFE answer was received by the USCIS on Sep 27
I-140 approved on Nov 06
I-485 approved on Nov 15

From what I know, this is not a world record (I read about an EB2 case approved in about two month) but it is certainly in line with what I was told last year (and, frankly, could not believe). I believe thousands of applications filed this last summer will have an adverse impact on processing times for those of you filing in the nearest future. Nevertheless, as long as you are originally from Europe I would estimate your GC chances favorably.

Card production ordered

2007-11-15T10:16:00.000-08:00

*** DO NOT RESPOND TO THIS E-MAIL ***

The last processing action taken on your case

Receipt Number: SRCxxxxxxxxxx

Application Type: I485 , APPLICATION TO REGISTER PERMANENT RESIDENCE OR TO ADJUST STATUS

Current Status: Card production ordered.

On November 15, 2007, we ordered production of your new card. Please allow 30 days for your card to be mailed to you. If we need something from you we will contact you. If you move before you receive the card, call customer service. You can also receive automatic e-mail updates as we process your case. Just follow the link below to register.

If you have questions or concerns about your application or the case status results listed above, or if you have not received a decision from USCIS within the current processing time listed*, please contact USCIS Customer Service at (800) 375-5283.

*Current processing times can be found on the USCIS website at www.uscis.gov under Case Status and Processing Dates.

*** Please do not respond to this e-mail message.

Sincerely,

The U.S. Citizenship and Immigration Services (USCIS)

TDD

2007-10-31T11:07:00.000-07:00

In my opinion test-driven development and refactoring are the most important contributions of the Agile movement. Personally, I find the movement itself rather controversial and oriented on less complex systems. I believe that school of thought brings in a few valuable ideas from cognitive and industrial psychology. On the flip side, I am highly suspicious of their lack of engineering rigor.

Throughout my career I have seen different development organizations. Some took unit testing for granted (sadly, TDD was not explicitly encouraged but even sufficiently extensive unit tests go a long way). Others only briefly dabbled with JUnit. The latter invariably turned out to work on profoundly boring applications with little to brag about in the quality attributes department. Naturally, this experience taught me to screen potential employers by their attitude to unit testing and test coverage tools.

Most of the time the refactoring part of the equation was mostly the matter of taste of individual developers. It is easy to justify but certainly much more challenging to amend. I think refactoring is so intrinsically intertwined with artsy questions of style, readability and proper OOD that it is destined to be a way of thinking and not merely a cookbook. If you remember the founding book on the subject, virtually each and every refactoring there is accompanied with a counterpart reversing the action - you choose which one to apply in a particular situation.

In my experience things that are difficult to formalize are easier to learn by watching. Probably this is the only way and that's why it takes so long for a developer to mature despite a relatively narrow skill set required typically. Such learning tends to evolve through time and requires good reference materials to learn from.

I am reading a couple of books on the subject, namely Test Driven and xUnit Test Patterns. Clearly, they are not comparable - the former is an also-ran, the latter is essentially the bible from a world-renowned series.

From Test Driven I would recommend an interesting chapter on testing concurrent code (thread-safety, blocking operations, starting and stopping threads, asynchronous execution). I hesitate to come up with another book addressing it.

Strangely enough, I found Test Patterns to be rather boring. Probably because of its 950 pages :) It reads more as a catalog of test idioms ("five ways to create a test fixture - transient, persistent, shared ..") and I was overwhelmed in my attempts to find and learn quickly a few new useful tricks. Surely enough the book is valuable, especially as a conceptual guide, but do not expect much fun.