Innovation 101
21 Oct 2013

If you can't rely on your eureka moment — and you shouldn't — then what's left to do to produce some actually innovative work? First, you need to admit something.

You suck

Your genius idea is most probably not new. You are working on a subject that has interested hundreds, thousands, perhaps billions in the history of science and humankind. You are not more clever than all of them. In fact, you probably sit somewhere in the last 10% in terms of intelligence and experience. The probability that you have all of a sudden thought of something that not a single one of them has already explored is infinitesimal. If you decide to pursue your initial idea anyway, you will eventually realize that it either sucks, or it has been done before. Most probably both.

However, what is possible, and in fact quite likely, is that you may have combined multiple ideas and concepts that did not originate from you and the result of this combination might be innovative. If you're lucky, then this combination might even prove to be powerful, in which case you will have made a breakthrough.

What I'm saying here is nothing new, of course.

Standing on the shoulders of giants

The wisest of the philosophers asked: We admit that our predecessors were wiser than we. At the same time we criticize their comments, often rejecting them and claiming that the truth rests with us. How is this possible?" The wise philosopher responded: "Who sees further a dwarf or a giant? Surely a giant for his eyes are situated at a higher level than those of the dwarf. But if the dwarf is placed on the shoulders of the giant who sees further? ... So too we are dwarfs astride the shoulders of giants. We master their wisdom and move beyond it. Due to their wisdom we grow wise and are able to say all that we say, but not because we are greater than they.

Isaiah di Trani (c. 1180 – c. 1250), source: Wikipedia, emphasis mine

No innovation ever happens in an isolated, confined environment. Or perhaps it does, but only in the minds of the most incredible geniuses. And no, I'm not referring to you.

We are all dwarves: our experience of life, the world and science is limited. However, what we can do is study the works of our giant predecessors. If we do, we will find their limitations. We will understand what makes them great, and what can make them even better. Their limitations are your opportunity for innovation. And you will address these limitations by looking for novel ideas, things that the original inventor did not, and could not know about. Perhaps the combination of ideas will not completely work out: again, this will be an opportunity for you.

You might fear that your peers will tell you that you are doing "incremental innovation". This will happen, no doubt about it. But it will happen very rarely if your work is sufficiently powerful. Nobody worth listening to will criticize the lack of innovation of a method that dominates benchmarks and opens the door to a whole new array of applications.

Conclusion? Here we go:

  1. Do your homework and see what others have tried before you.
  2. Pick a problem your own size.
  3. Have great results.

As a #4, I could add "Get ready to defend yourself against aggressive criticisms", but that would take a whole new post.

There is no genius idea
11 May 2013

This is the first of a series of blog posts on innovation and the research process. Most of these ideas gradually came to me, first while I was researching for my PhD, then as the need emerged for the design of innovative algorithms in the couple companies I worked.

How do ideas emerge? I bet you can think of a dozen times when you were thinking hard at a problem, and then you jumped up yelling: “Eureka! (or something along those lines). I do. I have seen it happen with countless people in many different situations. And sometimes you are so enthralled with that great, promising idea that you start working on it right away. That’s pretty much the definition of a “genius idea”. And in the excitement, this idea is utterly perfect.

But of course, this perfection is only an illusion. An idea, however great, is bound to be perfected in some way. And this is the gist of this great passage of Doctor Faustus, by Thomas Mann. This passage is literally at the centre of the book. It is a dialogue between Belzebuth and Adrian Leverkühn. The devil is convincing Adrian to use demoniac inspiration for musical composition.

“Let us just for an instance take the ‘idea’ — what you call that, what for a hundred years or so you have been calling it, sithence earlier there was no such category, as little as musical copyright and all that. The idea, then, a matter of three, four bars, no more, isn’t it? All the residue is elaboration, sticking at it. Or isn’t it? Good. But now we are all experts, all critics: we note that the idea is nothing new, that it all too much reminds us of something in Rimsky-Korsakov or Brahms. What is to be done? You just change it. But a changed idea, is that still an idea? Take Beethoven’s notebooks.There is no thematic conception there as God gave it. He remoulds it and adds ‘Meilleur.’ Scant confidence in God’s prompting, scant respect for it is expressed in that ‘Meilleur’ — itself not so very enthusiastic either. A genuine inspiration, immediate, absolute, unquestioned, ravishing, where there is no choice, no tinkering, no possible improvement; where all is as a sacred mandate, a visitation received by the possessed one with faltering and stumbling step, with shudders of awe from head to foot, with tears of joy blinding his eyes: no, that is not possible with God, who leaves the understanding too much to do. It comes but from the devil, the true master and giver of such rapture.”

Doctor Faustus, Thomas Mann, Scribd

So what’s new here? Everybody knows that work is 1% inspiration, 99% perspiration. But most people mistakenly believe that the effort consists only of implementing the idea: “All the residue is elaboration, sticking at it”. And that’s wrong, of course.

This passage dispells the whole concept that a fantastic idea is an end in itself. It also means that you should not become too dearly attached to great ideas. Instead, you should be ready to amend them, and sometimes to depart from them entirely, to “remould” them.

So, what’s the use of genius ideas? Personally, I have come to believe they are useless. Contrary to common belief, they don’t help jump start a business, and they don’t help overcome the competition. That will be the topic of the next blog post.

Self-hosted email
20 Mar 2012

As I explained in a previous post, I have decided to move away from Google's Gmail service for email management, and from third-party email hosting platforms in general. This isn't really a great accomplishment, and I am not trying to brag about it, nor to convince anyone that they should make the same decision. But a handful of people have shown interest in the method and the attached costs. And in my close circle, a handful of people who show interest in computer stuff is an awful lot. So here we go.

Overview

My setup is composed of three main components:

  1. A remote server that serves both as an SMTP server (for sending mail) and as a POP3 server. I pay 1€/month for this (see below for the financial details).
  2. A server which I own that retrieves the emails from the POP3 server (with getmail) and stores them in a maildir. Dovecot is an IMAP server which can serve my email to just any client.
  3. In particular, Dovecot serves my email to a webmail called Roundcube, also hosted on my server, and which serves as a replacement for Gmail's web interface.
Self-hosted email overview

Remote SMTP/POP3 server

Friends had warned me that managing an SMTP server was a royal pain in the ass. In particular, you need to pay attention not to be blacklisted by any large email delivery platform, such as Gmail, Hotmail, etc. So I decided early on I was ready to pay for this service. It just happens that Regfish (which is also my domain name provider) sells some cheap email packages for just 1 euro per month. With this service come a couple pretty classic, but very useful services:

  1. Catchall email addresses: that means that whatever gets sent to (where behmo.com is my domain name) will land in my inbox. That allows me to never give the same email address to two different online services. As a result, I know who sold my email address to spammers and my identity cannot be cross-referenced by multiple service owners.
  2. 100Mb remote mailbox equipped with webmail. If, for any reason (fire, apocalypse, reboot), my own server falls and stops retrieving email, my emails will not be lost and will be stored in a reasonably sized (100Mb) email account. That is, until my POP3 client wakes up again and catches up with the lost time.

All in all, Regfish provide a reliable service. I have been one of their clients since 2005 and it has been a pretty uneventful ride since then (which is a good thing, as far as server and domain name hosting go).

Local Maildir/Dovecot (IMAP) server

Of course, the whole point of this blog post is to demonstrate how you can self-host your emails, so it would not make much sense to keep them stored on the remote server, right? What moves them from Regfish's servers to mine is a cronjob started every two minutes that makes a call to getmail. Getmail is a basic Unix utility to which you feed a simple cnfiguration file where you specify: the address and credentials of the remote POP3/IMAP server (in our case: POP3, as we don't want the remote server to keep a copy of the emails), and the local folder where you want your emails to be stored. In this folder, each email is stored as a plain text file, and subfolders define labels. That also means that it becomes very easy to backup your emails, but this part will come in a later post.

Everything has been relatively easy until now :) No, seriously, getmail, the cronjob and maildir are all a piece of cake to configure. You can try them right away with any third-party email hosting platform that provide a POP3 interface, such as Gmail, Hotmail or Yahoo! Mail.

The Dovecot part is tricky though. Documentation is sparse, to say the least, and strongly depends on your Dovecot version. I think that wikis are just a poor choice when it comes to documenting software or code, but that's just me. It's too bad, really, because Dovecot is supposed to be the best of its breed. Anyway, I won't be able to help you with the Dovecot configuraton, which strongly depends on our platform, but you should make it if you read carefully the documentation included with your configuration file.

Roundcube webmail

I like my emails in a browser, not in a program such as Thunderbird or Outlook. I have looked long and hard for an alternative to Gmail's sleek interface (believe me, it has been long and it has been hard). Alas, the best solution I found is Roundcube, which is also the first result returned by Google when you search for "open source webmail". It's ugly, it's slow, it was coded in PHP, it doesn't support CardDAV for contact sync, but it works. Which is always better than most other solutions I tried. Install is easy, configuration and use too.

Conclusions

The whole thing works, and better: it is very robust and fault-tolerant. The only critical moving part that may not be unplugged is the remote mail server. If it fails, I won't even know it, except that certain mails will not arrive anymore. But that has never occurred until now. As I emphasised earlier, security of my email data is paramount and in this matter I have not been disappointed until now.

The only problems that I see with my setup are the lack of a dynamic, responsive webmail interface (I have even considered coding a better one myself), and of an integrated contact synchronization solution. Funambol works well in itself, but does not get along well with Roundcube. I keep looking.

Naturally, this installation has a financial cost. My personal server is a low-power computer that has been plugged at home 24/7 for the past year. Its construction cost was ~450€, but since I use it for may more things than just email, I consider that its cost has already been amortized. It draws ~30W, and in France that represents a recurring cost of about 3€/month. But then again, this server would stay on even if did not host my email. Finally, there is the cost of my Regfish email account: 1€/month. But now that I think of it, I could probably avoid it if I used my Free account that comes with my home internet connection

Bye Bye Gmail
23 Feb 2012

Since a couple months ago, I have stopped using my address and have now replaced it entirely by my new one: . I think this is worth an explanation.

I own my address

First of all, I do not wish to be tied to an email address which I do not own. As a reminder, all @gmail.com addresses are owned not their users, but by Google. This increases the cost of switching email address: if your email account is disabled, you run the risk of losing contacts who are not aware of your address change. This is similar to changing your mobile phone number; usually, what you do is that you send your close friends your new phone number. Naturally, notifying all of my 2400 email contacts of an address change is not an option. So I decided to redirect all Gmail-incoming emails to my newly acquired @behmo.com address and to send all emails from this new address.

I own my data

But I also decided to move my data away from Gmail. This has been a tough decision, technically speaking. I was one of the very first Gmail users, back in 2004. My main Gmail address now hosts 6.2 Gb of emails. Around mid 2011, I realised how important to me was the content of my mailbox: it contains all my contacts, all of my intimate correspondence with my family, all of my love affairs, in-depth reflection with my advisors about my PhD, a lot of photography work, bank account coordinates, clear-text passwords from various websites, a small amount of illegal music files, professional correspondence with potential or actual employers, and much more. Losing all this data would be dreadful. And you know what? it happens. Worse, sometimes Google makes it happen: it has happened more and more frequently with the rise of Google's social network Google+ and its requirement to make use of the user's real name. And for different reasons, I do not want to use my real name on Google+. Losing the content of my mailbox was not, and still isn't an option, so trusting Google with it has become less and less rational.

I have nothing to hide, but my friends might

For all these reasons, I am now self-hosting my email on my personal server, of which I make frequent backups. The technical and financial details of this move will be given in later posts. I would just like to mention one last argument which has been decisive in my choice of switching to a self-hosted email service: I am concerned not only by the safety of my data, but also of my friends' and family's. Suppose one of my friends commits a crime and, for one reason or another, tells me about it in an email. He might need help or just need to talk about it. This email becomes a piece of evidence which can be used against him. In the past, Google, Yahoo and Microsoft have all complied with police warrants from various countries to provide personal user data. This situation has made me more and more uncomfortable, if not downright anxious. They tell me I have nothing to fear if I have nothing to hide. Well, I know about me, but what about my friends?

Low-Power, Silent Home Server Build
20 Jul 2011

Recently, I decided I needed a home server to be left switched on 24/7. I wanted to use it both as a media (music & video), web, ssh and samba server, and a torrent client. It had to be a low-powered PC, because I have to pay my electricity bills every month. It also had to be as silent as possible, because nobody loves a deep humming in the background during a romantic dinner. After consulting with the wonderful folks at silentpcreview.com, I ended up with this configuration:

(Notice the fanless motherboard and power unit)

As for the power consumption... 29W! That number made my day when I first plugged my wattmeter to my server :D

Finally, here is the software stack that I use:

  • My beloved Ubuntu 10.04.2 LTS, because this is a headless server that needs stability before all things. I tried out Ubuntu 11.04 and it made a few impressive crashes right after the first install.
  • Apache HTTP Server, because it's the only web server I feel confident I can handle.
  • Subsonic, which is the only decent music streaming server I have tried.
  • rTorrent inside a screen. Yes, I download stuff.