Productify

I think a good way of learning something well, is to try to expand upon what you want to learn, and make a product out of it. Instead of wanting to learn about X, looking up X in Wikipedia and be done with it the next minute, how about spending some time figuring out something you can actually create once you know X?

At least for me, instead of learning a bunch of disparate things separately, it's much more motivating to put things in a larger context and actually build something tangible out of it.

And by product I don't necessarily mean something you can earn tons of money from. Of course it's nice if your product can make you a gazillion dollars, but it's ok too to make a product that noone wants to pay a dime for. (I'm from a socialist country!) As long as you are having fun and learning something during the process, it's all good. I believe all learning has value, strengthening you and increasing your chances of favorable positions down the road.

To create a product means to identify a chain of sub-problems that needs to be solved, deciding on an order in which to solve them, and then solving them one by one. Some sub-problems may require you to look things up and learn (it's ok to use Wikipedia!), and some sub-problems are things you need to invent, or just execute. Some of the sub-problems are obvious when you think of the product in the first place, but I believe most products also have a bunch of hidden sub-problems. Problems that you don't come across or imagine unless you actually start breaking things down and get working on making the product. Stumbling upon and solving those problems are valuable too, sometimes more so than working on the most obvious ones.

Besides from learning many new things from solving these sub-problems, creating a product is a nice feeling. You can show a product to others as a tangible demonstration of your skills and creativity. Perhaps you can even sell the product and earn money from it. Or if not, perhaps it spawns ideas for new products that will. (Then go make those!)

So how do you start? Which product will you make? Perhaps you don't have a killer idea for a completely new revolutionary product that will rock the world. It's ok. I know a fun exercise: Think of an existing product and try to figure out how it's made. Which problems need to be solved to make product X? What do you need to learn, and which things do you have to create, and in which order? As a next step, how about seeing if you actually can make it? Perhaps you're not able to set off enough time to create say a next-gen particle accelerator, but hey if you look around there are tons of smaller products that could be fun to emulate and learn from. Be kind to yourself and let the definition of product encompass things that can be made in one or a few hours.

So, try to come up with a product. Or steal one.

Eventually when you're finished making the product, write about it. As they say, "learning by teaching" is perhaps the best way of learning. By figuring out how to teach something in a clear way to someone else, you force yourself to organize your thoughts. You think about which are the important parts and which parts can be left out. You think about in which order to present all the sub-problems. You imagine questions, forcing you to really think things through and not brushing over any important details. Also it gives you valuable writing practice.

And once you're finished writing, you know what? You just got yourself another product. Because your writing is most definitely a product too. Hey, now you have two!

To summarize:

1. Think of a problem or something you want to learn
2. Make a product out of it
3. Write about it

Simple, isn't it?

A particle accelerator. You know, for inspiration. No pressure!

Node.js

Admittedly, besides from a little dabbling with PHP a long time ago, I don't have much experience in writing server side applications in other frameworks than Node.js. So I don't have much to compare with. But I have to say I'm surprised by how quick and easy it is to write server side apps using Node.js.

For those who don't already know, Node.js is a software platform for server side apps, and it is built upon the same Javascript engine (called V8) as used in Google Chrome. Thus Node.js makes it possible to execute Javascript code on the server. Before Node.js, Javascript was regarded solely as a client-side language.

Perhaps the first benefit that comes to mind with this is that we can now share code between the client and server implementations. Let's say you are making a simple multiplayer game in Javascript, and you want the server to be in charge of the game logic, but you also want clients to run game logic to be able to predict the game state while waiting for updates from the server. Without running the same language on server and client this could be quite a hazzle, and even small differences between the client's and the server's engine state calculations could lead to large confusing differences in state. Thus it's a really good advantage to be able to run the exact same logic on both sides. (Besides, obviously you wouldn't have to implement the game engine twice.)

This leads us to another aspect of Node.js: Node.js apps are typically single-threaded, meaning one single thread on the server handles all the requests made from multiple clients. Client-server work should be designed to be asynchronous, i.e. typically the client sends a request to the server and the server will call the client back shortly later using a callback function. As a benefit of being single-threaded, the overhead needed to serve a new connecting client is very low: the server does not need to create any new process or allocate any new memory, and it does not have to do any context switching to serve multiple clients.

As a result, one server app can typically handle a large number of clients without bogging down. On the other hand, this means that it's bad to spend a long execution time serving one client as work will not be going on in parallell. Going back to the multiplayer game example above, it's then actually a bad idea to run game engine code on the server unless the executions can be made quickly. I guess Node.js apps should be designed to do just short operations. (I have heard there are some attempts in using something like webworkers to spawn threads for longer running operations, but I have not tried it out.)

Also, serving clients in one process is obviously dangerous if some code is misbehaving: if one operation crashes, the whole server app goes down. As an example, I found out that the file system call to read from a file will throw an error if the file does not exist. Forget to write exception handling for that, and you'll watch your server go down in flames when one single client cannot find the file it is looking for.

Anyway. For my experiments so far Node.js has been nice to work with, and the benefits have greatly outweighed the limitations. Knowing a bit of Javascript it's usually very quick, and does not require much code, to achieve quite a lot. There are tons of open-source packages you can use to speed up your work, and a package manager (aptly called NPM, Node Package Manager) to help you installing things.

So far, using Node.js, I have written:

• One file server, which I use as a web server on my Linode machine, i.e. to serve files over http at www.mattiaserlo.com
• One "session server", which uses socket.io to handle communication between clients. I'm using this for simple multiplayer games and chat-type apps.
• A few RESTful services, that I used for testing to store and retrieve data from the server.

I'm running all of those on my rented Linode server.

To get started using Node.js on a Linux machine, go to http://nodejs.org/ and download an archive, then unpack and install like this, using shell:

tar xzvf node-v0.6.7.tar.gz (or whichever version you have downloaded)
cd node-v0.6.7
./configure
make
make install

You can run Node.js apps on your local computer using localhost, although it's much more fun to host your stuff on a real server that is always accessible by your friends. If you don't already have access to a server, I recommend renting one: see my earlier posting, http://mattiaserlo.blogspot.jp/2013/06/virtual-private-server.html

Finally, as an example, let me show how easy it is to create a file server using Node.js and a few packages:

Assuming you have installed Node.js (see above),

1. Do "npm install node-static"
2. Do "npm install http"
3. Create a file named myfileserver.js, containing this:

var static = require('node-static');
var http = require('http');
var file = new(static.Server)('/var/www'); // The folder which you want to serve files from
var httpServer = http.createServer(function(request, response) {
request.addListener('end', function() {
file.serve(request, response);
}
}).listen(80); // The port number your server will listen to requests on

4. Run the server like this: node myfileserver.js

Now you can connect to your server from another computer and fetch files from /var/www.

To run your fileserver as a service, instead of from a blocking shell call, break the Node.js execution above and create a file named myfileserver.conf, containing this:

description "My node.js server"
author "Me"

start on started mountall
stop on shutdown

# Automatically Respawn:
respawn
respawn limit 99 5

script
export HOME="/root"
exec /usr/local/bin/node /node/myfileserver.js >> /var/myfileserver.log 2>&1
end script

post-start script
# Optionally put a script here that will notifiy you node has (re)started
# /root/bin/hoptoad.sh "node.js has started!"
end script

The exec line above assumes you have stored myfileserver.js in a folder called /node.

Store the conf file in /etc/init on your machine, then you can start and stop your service whenever you want, like this:

start myfileserver
stop myfileserver


Virtual Private Server

One of many things I've been wanting to get more experience in is web development. A few years ago I did a simple Tetris clone which was using Bluetooth to exchange game data between two mobile phones, and now I wanted to do similar things but using Internet. I also got interested in Node.js, having read a lot about that and Socket.IO, and so I decided to try renting a virtual private server for some experimentation.

After a very quick research I decided on Linode, www.linode.com. Even though Linode does not have any free plan, it does give you quite a lot: you get your own static IP address, you can choose which OS to install (I chose Ubuntu) and you can run whichever long-running services you may want. It's pretty much like having your own Linux server, only it's tucked away some place else than in your bedroom. (VPS is a very fitting description.) You upload files easily using FTP and you use ssh to work with the shell. Given the great smartphones and tablets of today, you can even run ssh from your phone while on the go, and suddenly you have a powerful Linux server right in your pocket.

I went for the cheapest plan, which gives you:
* 1 GB of RAM (Used to be 512 MB but they doubled it without increasing the price.)
* 8 CPU cores (Though I guess this does not mean guaranteed exclusive usage.)
* 24 GB of storage
* 2 TB of transfer data (This is vastly increased from since I signed up.)

The price tag for this is 20 dollars a month, which I consider neither particular cheap nor very expensive.

My first plan was to just try it out for a month, do some experiments and then be done with it. However, that was one and a half year ago and I'm still paying. Actually it's so nice to have access to Linode, and I'm having so much fun putting up experimental apps there now, that I don't see any reason to quit. Maybe in the future I'll buy a real physical server, but for now I'm not ready to cough up money for that and be bothered having one more machine taking up space in my small apartment.
Also I'm very happy with the stability of Linode. I have not experienced any down time at all this far.

Cheers to Linode!

Oh and by the way, the picture here above I just Googled and I don't know which server hall it is from. I wonder if "my" Linode is surrounded by cool blue lights like that..?

Drive

I took a decision some time ago to try to change my role at work to be able to spend more time working actively with software again.

I don't mean only programming, but learning more about software and code quality, frameworks, patterns, languages, algorithms and data structures, operating systems, trends and so on. And yes, programming too.

I believe I used to be a good programmer before that point in time, but the way things had developed at the company I found myself in roles where I did a lot of coordination and management-like work for a couple of years, gradually losing engineering competence from not doing enough of the technical stuff.

You know, the fun stuff.

The thing is, if a tech company would be a racing team, I think of the software engineer as the race car driver. Even though the outcome of course depends on a large group of people in the team, all with different roles, it's arguably the driver that has the most exciting job. And it's the driver that brings the car over the finish line. Although I appreciate skilled people in any role, it's the talented driver that earns my highest respect. I want to be the driver.

Yet after driving some races usually your company will want you to move on to do other stuff, like watching races and giving feedback to the new younger racers. And write documents about driving processes. And help driver A communicate with driver B in another country. And so on. Perhaps the company might even outsource all the racing! What's up with that?

Getting older, people around you may look up to you as an awesome driver, a driver with tons of racing experience. Yet you may not have driven any race in years. The thing is, maintaining skills as a driver requires that you actually keep driving now and then. Otherwise, as time passes by, one day you'll have forgotten how to do it.

My plan was two-folded: try to change my work role so that I would do less administrative tasks and spend more time doing hands-on technical work, and also to set off time at home during evenings and weekends to learn new things, by reading articles and doing experimental software projects as a hobby.

And the winnings I imagined would also be two-folded: I would have more fun, and I would be better prepared for change because I would be technically stronger.

It's good to be prepared for change. Even if you don't have a plan to change jobs anytime soon, the fact is you don't have absolute control over the company you're working for - no-one does - and so you really don't know whether you might be forced to change. And if you need to change, be it for internal or external reasons, most likely you'll need to go to interviews. Are you prepared to go on an interview and explain in detail what you have been doing the last two years and which parts of product X you have contributed to, and how? Later being asked to write source code on a whiteboard to solve a technical problem, in case it is a software engineering position. I went to an interview and failed, realizing far too late that I had fallen behind technically. Not from not taking work seriously at my workplace, but because I had done exactly everything my company asked me to do, and not all of those things were strengthening my profile to prepare me for other roles in other companies. Actually several of the things I had been working hard on were rather counter-productive, distancing me from the profile I wanted to maintain. And I had failed to see that I needed to compensate for this by spending free time to actually stay up to date and skilled.

This was a late realization for me, that I needed to reflect more actively on where I was going and whether I was ok with the direction, and whether I felt I was learning enough. Not to mention whether I was having fun or not. Work is supposed to be fun. And learning should be fun.

Shortly thereafter, with my new approach, I felt joy again. Joy to learn new things. Joy to relearn things I had been studying or practicing long before. And most of all, joy to do what got me interested in computers in the first place: creating cool things with software.

I also changed the way I see "gaps" in my knowledge. Whereas before I would find a gap and be disappointed that there was something I didn't know, I decided to appreciate gaps as they are opportunities to learn something new and improve my skills. Moreover I started seeking out gaps, trying to track down things I felt I ought to know about but yet didn't. It's much better to find gaps on your own in a controlled environment than in a future job interview. And it's fun to feel how you are progressing and becoming better at something. I had almost forgotten that.

So, be prepared, and get behind the wheel and drive now and then. If not in the sense of doing hands-on software engineering, in case that's not your cup of tea, then in the sense of having control over where you are heading and your self development. And enjoy the ride!

Umeshu

It's finally time to do something I've been wanting to do for a long time: my very own umeshu!

It doesn't seem very hard. All you need is some plums, some sake and a bunch of sugar. Oh and patience, maybe that's the most important ingredient. Because you need to wait for around two months before it's ready for you. Or six months if you want it to be even more tasty. But I'm not sure I'll be able to wait that long. In fact, I'm sure I'm not.

Get the ingredients...

...put the plums in a jar...

...add sugar and sake...

...put a lid on and wait.