Theo Schlossnagle is of the view that developers need to take the operational view when writing code. It's a very valid point of view and something that I suspect most of us overlook.

I'm going to expand on what he said and share my ideas on that.

Let's take the software you're writing now (assuming it's web software). Is it operable?
It probably does atleast these things

• Fulfills your requirements document
• Passes your unit tests and integration tests
• The UI is usable and responsive

But is it operable? Once it's deployed, can it survive unprecedented load? Fringe cases? Subsystems going down? Third party services it depends on becoming unavailable?

And present a front of graceful degradation as it does all this?

Selective Failure

Most of the time, we stress systems before deployment, using load tests to simulate real world conditions. That takes care of one aspect. But most of us don't think of failures of selective systems, especially when the system is distributed and its components interact in complex ways. The latter is true of most big web applications.Handling selective subsystem failures is not purely an operations responsibility. The application has to be written keeping selective failure in mind.

In the video, Theo brings up an analogy with security. Security is not a feature, but a way of thinking.

Sanitizing user input before putting it in a database query is not a feature.

Not allowing access to your internal web services is not a feature.

These are security restrictions you automatically think of when you develop.

In the same way, operational thinking should not be a something postponed till deployment while writing code. It should be de rigueur in the design and development process.

Let’s take the example on the puzzle page –

TIHS SENTENTCNES ISS NOUT VARRRY GOUD

It’s a munged up version of

THIS SENTENCE IS NOT VERY GOOD

The original problem statement involves minimizing the score (number of changes necessary) to transform each word in the munged up sentence into words that are in a predefined wordlist. It says nothing about grammatical correctness of the result.

So going by that, the score and the output sentence from my implementation are respectively, 8 and

TICS SENTENCES IDS NOT VAGARY GAUD

From the standpoint of the original problem, this solution is correct.
But not from a grammatical point of view. After some poking around, I realized it’s not possible for this program to spew out the correct version of the sentence without it being aware of English grammar rules.

P.S. Without getting into grammar correction, of which I have no idea how to implement, I made a small change in the program as an experiment. The second version favours a smaller word whenever the scores for two are equal.
And the result is (Score remains the same)

TIS SENTENCES IS NOT VARY GOD

Have decided to put the solutions to the exercises I’m doing here.

-- File httpd.c
#include "../mynet.h"

if(errno = EINTR) {
    //do something
} else {
    err_sys("read error")
}

Yes it’s a stupid beginner mistake – typing the assignment operator instead of the equals check. The thread of execution would never enter the else block. I corrected it, but the interesting part came when I tried to compile it.

cc ../mynet.c httpd.c

mynet.c contains some handy helper functions that I’ve used in my other server classes. Guess what – the compilation failed with this message

"httpd.c:(.text+0x6a): undefined reference to `err_sys'"

I checked my header and the err_sys function was nowhere to be seen. If this function is missing, how did my other class (from where I copied this code) compile previously?
After some fiddling around I put the assignment operator bug back, and guess what? The code compiled fine.

Based on just these observations, we can conclude that the gcc C compiler ignored the unreachable (else) part of the code. It did not even check if the code inside the else block was legitimate. How far did this behaviour go? Let’s see.

-- File httpd.c
#include "../mynet.h"

if(errno = EINTR) {
    //do something
} else {
    mocha(); //Undefined function
}

This compiles fine.

-- File httpd.c
#include "../mynet.h"

if(errno = EINTR) {
    //do something
} else {
    asdf;
}

This correctly fails with an error.

So syntax checks are being done in code that is known to be unreachable, but there are no checks for undefined functions. A bug? I would say yes. Google did not turn up much except this old link – http://compgroups.net/comp.lang.c++.moderated/could-if-else-avoid-syntax-checking-compile-time-unreachable-code

Unfortunately I’ve been able to devote time for this mostly on the weekends. This translates to slow progress because UNP goes into a lot of depth about everything. This is a good thing, but it also means that I’ve to reread and review the last few pages everytime I try to pick up where I left off. This does not really help when I’m trying to understand concepts in depth. So what’s the solution? I’ll try to do a bit atleast 3-4 days a week from now on.

UNP is an amazingly detailed book – and as one of my colleagues said – “If you read that book properly, Stevens makes sure that there’s nothing left for you to know on the topic”. I agree.

Stevens wrote his own wrapper functions over common socket functions and used them in all the code examples in the book. The wrappers handle all error codes and portability issues (like IPv4/IPv6). These are included in a header unp.h (available in the back of the book as well as online on http://www.unpbook.com/src.html).

Some reviewers of the book gripe about this and say that this is an obstacle to learning the actual functions. But I think that there was no other way to do it without littering every example snippet with code for portability and error handling. The wrapper strategy makes it easier to follow the examples, and at the same time – as I found out – it makes you write those wrappers yourself. True, you can just include the unp.h header as you try the examples, but then you’ll never know what those functions are doing. I’ve found that creating my own header and writing the functions as I come across them, after looking at the book’s source code, works great. Most of them will end up identical to those in unp.h.

I’m pushing the examples I’m trying out into github – it’s a scratchpad so not everything might compile.

I’ve added a generic startserver function to my header – this takes a pointer to a function as an argument. The generic function starts a server socket (bind/listen/accept), forks a child when a client connects and calls the function that was earlier passed as an argument, abstracting out the actual serving part. The function pointer syntax was not hard to figure out – I’d read Peter van der Linden‘s algo on unscrambling declarations in C last week. Interesting how things add up!

Fast forward 8 years. The products I’ve worked on till now have been in Java. Serverside Java, Infrastructure Java, Applications Java. An appserver, a desktop app, a SaaS app. Abstractions, APIs, SPIs, Specifications, Javadocs, Objects talking to each other – it’s a satisfying, pleasant world if you like high level order.

I made occasional forays into Python and Ruby for some side projects and scripting.

But lately, I’ve been wanting to code in a really low level language. Yes, like C. Let me rephrase that. I’ve been wanting to write code where I can be as close to the OS as possible, learn bit twiddling hacks, make low level system calls. I’m not sure what this desire was born out of. Maybe 8 years of Java. Maybe wanting low level control over what I was doing.

So I’ve ploughed through K&R the last couple of months. Bought Deep C Secrets. Regrokked pointers – properly, this time. Had a brief period of not knowing where to go after this, which resolved itself in the form of Unix Network Programming (Richard Stevens). I’m working through Vol 1 right now. It’s slow going at times, but I’m loving every moment. I’ve fallen through – the abstractions, the bytecode, the layers of objects – and reached the ocean floor.

<In the background, the Toccata ends, and the Fugue begins.>

Think again.

Most of us use one of these -

1. Facebook
2. Twitter
3. ShareThis
4. Technorati/Digg/et al

There’s a common aspect to all these networks/tools – all of them can potentially collect data about the online preferences of their users. So – do they? Some of them do.

Online preferences are links that you visit, which translates to things that you are interested in. This kind of data can be used to build up a profile of the user.

Think about it -

1.  Facebook knows what you share on facebook.com, knows what you “Like” among others’ shared links, and now with OpenGraph knows what you “Like” on sites that have the Facebook Like button.

2.  Twitter knows what links you share, and now with t.co – its own shortening service – it will know what links shared by others you clicked on (read “interested in”). From a Twitter blog post -

routing links through this service will eventually contribute to the metrics behind our Promoted Tweets platform and provide an important quality signal for our Resonance algorithm—the way we determine if a Tweet is relevant and interesting to users

3. ShareThis – if you’re logged into ShareThis, it knows what you shared.

Links you share and visit provide a picture, albeit incomplete, of your online preferences.

The question is, how are these tools and services planning to use this data?

If you know what someone likes, you can recommend stuff to that person. A lot of sites do this already. These recommendations are based on multiple parameters. E.g. Amazon’s recommendation system – which does a great job – uses collaborative filtering. Simply put, it uses data from your past purchases, ratings and I-Own-This history and from other users whose history is similar to yours. The more history you have on Amazon, the better your recommendations get.

Building a content recommendation system seems to be an obvious step once you have a data mountain of your users’ likes. And this is what these sites seem to be doing but to achieve different ends.

E.g. Facebook – See slide #29 http://www.slideshare.net/CMSummit/ms-internet-trends060710final. This has not happened yet, but what’s  stopping it, considering what Mark Zuckerberg said earlier this year ?

Twitter has recommendation plans – http://groups.google.com/group/twitter-development-talk/browse_thread/thread/14d5474c13ed84aa?pli=1

ShareThis already has behavioural advertising in the works with its segmentation technology.

The bottom line is – some of these services are going to use it to improve the end user’s experience – and will do so within the boundaries of their privacy policies. The rest – we don’t know.

Code Snippet 1

public class Init {
    {
        System.out.println("In the beginning was the command line");
    }

    public Init()
    {
        System.out.println("Created an instance");
    }

    public static void main(String[] args)
    {
        Init init = new Init();
    }
}

What do you think the output is? It’s this -

    In the beginning was the command line
    Created an instance

The ‘hanging’ braces at the start of the class definition are instance initializers. Most of us are more familiar with static initializers -

Code Snippet 2

static
{
    //Do stuff
}

Instance initializers (II) are not seen often in everyday Java code – so they might seem odd at first. They are executed every time an instance of that class is created, before the statements in the constructor are executed. (See The Java Language Specification 3 section 8.6).

One use of IIs can be to execute something whenever an instance is created, and the class has multiple constructors, without calling it in every single constructor.
Another one which has become popular is populating collections during declaration, in the style of Ruby or Python single-line initializers -

Code Snippet 3

private Set<String> names = new HashSet<String>() {
    {
        add("Rigel");
        add("Vega");
        add("Antares");
    }
};

This idiom was how I encountered IIs first while reading somebody’s blog.
What is actually happening here?

1. An anonymous inner class is created.
2. An instance initializer block is added to the anon inner class.
3. Objects are added to the instance of that class when the names variable is initialized.

Now take this scenario
Code Snippet 4

public class WrongUsage {

    private Set<String> names;

    {
        add("pleiades");
    }

    public void WrongUsage()
    {
        names = new HashSet<String>();
    }

    public void add(String name)
    {
        names.add(name);
    }
}

Based on what we have seen above, the names set is used before it’s initialized. So this throws a NullPointerException.
Let’s take another case – similar to the above but involving inheritance.

Code Snippet 5

public class MyHashSet extends HashSet {
    {
        add("pleiades");
        System.out.println("Added");
    }

    public MyHashSet()
    {
        super();
        System.out.println("After calling super");
    }

    public static void main(String[] args)
    {
        Set set = new MyHashSet();
    }
}

This runs, with the output being

    Added
    After calling super

In this case, add() internally uses the inner HashMap inside HashSet which is initialized in the HashSet constructor. This implies that the instance initializer is invoked before the class constructor, but after the superclass constructor (The super call is redundant here. It will be called anyway).

So the sequence is

1. Superclass initialization (this includes superclass instance initializers and constructor)
2. Current class’s Instance initializers
3. Current class’s Constructor

This is why the code in Code Snippet 3 does not throw an NPE – because it’s a case of inheritance (the anon inner class is a subclass of HashSet)

Respond – on time, with a clear actionable, and follow up.

This post is about how not to do customer service – and I am going to take a recent bad experience with one of India’s top online stores – http://www.indiaplaza.in. They sell books, among a lot of other things, and I have been buying from them since 2007.

I had ordered 4 books from them last month. 3 of them were shipped on time. When there was no news of the 4th one, I checked the Pending Orders page. It was not updated and stated that the book will be shipped by x – a date 4 days in the past. “Ok”, I thought, “let’s contact them”.

I sent off a mail to their customer service ID – which I had been using earlier. An autoreply came back saying that they do not respond to queries anymore from that ID, and I have to fill out a form on their site. That would raise a support ticket.

Which I did. And the form’s response promised that I would get a response within 24 hours.
Which did not happen. And their phone number is hidden in a small Contact Us link – I did not find it.
So I raised another ticket after waiting for a day.

Nothing happened.

At this point I had no visible working way of contacting them (apart from the phone number which I could not find. I attribute this to the fact that the most prominent “Help” link on their site is “Customer Support” – which points to the form mentioned above. That is what most people would click on).

So I went and posted my case on www.consumercourt.in – a site where consumers can go and post their grievances. Apparently that worked – for within 4 hours I got a call from Indiaplaza about the non-availability of the book. There was no apology, though. They promised to deliver it after 7 days. Now, this was extremely surprising. It indicated that their support team checks online complaint sites for issues with Indiaplaza, but do not check their own support system!

Anyways, nothing happened after 7 days – so I went through the same ticketing system again. This time there was a delayed response (thank goodness!) stating that the book was not available. “Fine”, I said, “Just refund my money”. They agreed to do it within 5 days.

Nothing happened (Do you see a pattern here?) . So I raised another ticket…and so on.

Four things they could have done better

1. The deployment of a well thought out customer support system, which is convenient to use and not just built out of considerations like it’s easy to use for their support group or helps in cost cutting.
2. Responding to my query on their ticketing system within the promised time.
3. After not doing (2), followed by the incidents mentioned above, they could have taken extra care about this case. Once you piss off a customer, you have to do extra work to get him where he was before, and still more work to make him happy.
4. Build a better online buying system where the status of the order is updated automatically.

The feeling that I have as a customer that I am being ignored, and especially after I have been billed, with prior experience not helping me to restore trust, is a damning indicator of what Indiaplaza lacks. They have just lost an old customer, and with the power of word of mouth these days, a lot of potential future customers as well.

Update: After raising another ticket, I finally got my refund. But I am not going back there

Rosenberg follows the team very closely, participating in their meetings, interviewing them and filling the narrative with his own insights.

If a developer were to read this book, s/he would recognize the events in it this as standard stuff when developing a product, and s/he would also identify with the team itself, with the occasional shaking of the head

The software they set out to develop was based on Kapor’s vision of a game-changing PIM. They called it Chandler, after one of Kapor’s loved crossbred dogs.

This was the pre-Web 2.0 era, where most things, including product releases took longer to happen. The browser had not yet become the first medium for delivering any consumer facing application. Desktop applications for collaboration still looked as if they had potential.

Chandler’s ultimate aim was to be a all-in-one tool where users could access their email, personal notes, calendar events and to do lists through one interface. Kapor visualized all these different data collections as isolated information silos. Chandler would break open and combine them. It would let you view and move around all these disparate things in a unified manner. You could tag a note with labels, and then put a date on it and make it a calendar event, or send it off as an email. Grand vision? Yes. It’s something I would personally prefer to have around to manage things.

Due to numerous reasons, Chandler was late in delivering its promise. Gmail came in 2004, and opened the floodgates of what was possible using new Web technologies. Firefox arrived, and forced Microsoft to relook at IE (and make it better). Chandler was yet to release its 1.0 version. The browser was fast overtaking the desktop.

We have been spoilt by the seamless interconnectivity provided by the Internet – it’s always there – home or office or a mountaintop – and with it, our data. Chandler did attempt a Web interface where you could access your data (which would be synced between your desktop and a central server), but it was too late.

The Chandler project is still on with many faithful developers. I tried out the latest version after finishing the book. My impressions? It still has a long way to go, going by the number of bugs I encountered, and assuming it can convince the Web 2.0 crowd to switch to a desktop application by the sheer force of its design and the things it is supposed to do.

The most interesting chapter in the book is “Methods” – where Rosenberg does a brief and very readable survey of the evolution of development methods over the years. This chapter can be read standalone even if you don’t read the whole book.