Dear Uber: you’re great but please add this feature!

UPDATE: A few additions at the end. I was pretty surprised (and so was my WP server) that that this made HN front-page. Also – hello, Cloudflare!

When ordering an Uber ride, I (obviously) have to enter my pickup location.

Can I please also enter the intended dropoff location? Maybe like this:

set_dropoff

Or, maybe even better: let me enter it after the pick-up has been confirmed, while waiting for the driver. And, while we’re at it: let me also pick a preferred route, if I want to.

Why? Because that way, the driver will already have directions pre-loaded in their GPS when they pick me up. Makes is easier for both of us, and the whole experience becomes even smoother.

And, it will prevent this from happening again:

gotLost

That was earlier this year. Here’s what happend: the Uber app, which uses Google Maps, was able to find my destination address – I know this because I requested a fare quote. However, the driver used a different GPS for navigation, which did not find it. And the driver was unfamiliar with the neighborhood (and so was I).

This could have been avoided if the driver’s GPS would have been pre-loaded with directions.

By the way – why do drivers have to use the Uber driver app and a separate navigation device anyways? Add navigation capabilities to the app, and the drivers can ditch their GPS devices. And will also be less distracted while driving.

Sounds good? Great, let me know when it’s done. Oh, and thanks for the free black car upgrade yesterday. Keep up the good work!

Discuss on HN.

UPDATE: some interesting comments on HN, here are my responses:

  • Risk of drivers rejecting rides based on drop-off location (pointed out by tptacek, apaprocki and others): that’s a valid concern, but as jaredsohn points out correctly, this can easily be mitigated by hiding this information from the driver until they actually pick up the passenger.
  • Drivers using their smartphones for navigation could be less safe than dash-mounted nav devices (also pointed out by tptacek): true, drivers would have to be required to dash-mount their smartphones in a similarly fashion. This setup would certainly be much safer than the current one, where they’re paying attention to a mounted GPS and their non-mounted smartphone running the Uber app.
    Also – maybe at some point in the future, the Uber driver’s app can run directly on the car’s on-board computer system – now that would be nice, wouldn’t it?
  • dsl points out that “If you request a fare quote before your request, the driver will see your destination once they have accepted the fare” – now that is interesting! I’d suggest to make this more prominent to the user, and also add route selection (for the passenger) and gps integration (for the driver).

 

Dart + CI with a few lines of code

Dart not only runs in the browser, but also on the server. Just like JavaScript, since the invention of node.js. 

Here are two simple answers for two practical questions:

1. What’s keeping my Dart process alive?
2. How can I set up continuous integration?

Keep Yourself Alive – Keep Yourself Aliii-hiive …

Let’s say you wrote a social networking web site in Dart, like this:

 import 'dart:io';
 import 'dart:math';
 void main() {
   HttpServer.bind(InternetAddress.ANY_IP_V4, 80)
     .then((HttpServer server) {
       server.listen((HttpRequest request) {
         var n = (new Random()).nextInt(1000);
         request.response.statusCode = 200;
         request.response.write(
            "<html><body><h2>Welcome to THE-BEST-SOCIAL-SITE 2.0!</h2>
             You have $n friends.</body></html>");
         request.response.close();
 });});}

Done! Now let’s quickly copy the file to your server and spawn a new Dart process:

$ dart social.dart

Your site is live and millions of users have started flocking towards it… until your hosting provider decides to reboot your server and you’ll have to restart the process manually. Or, until an uncaught exception kills your Dart process.

Simple fix: let upstart manage your Dart process. First, install it – for example, in Ubuntu:

$ sudo apt-get install upstart

Now, create a new config file in /etc/init. For example, social.conf:

exec /usr/bin/dart social.dart >> /var/log/social/$(date -d "today" +"%Y_%m_%d_%H_%M_%S").log
start on filesystem and static-network-up
stop on runlevel [016]
respawn

You’re ready to go! As a test, reboot the server and check if your web site is up and running afterwards. Now that was easy!

Continuous integration – or not integration at all … ship-rooom-meetings – leave the hall!

Continuous integration (CI) is awesome. No need for painful integrations and deployments, just get your code out there. In fact, if your project is at an early stage, you might wanna just ship to your production server on every check-in. Well, maybe to a staging environment first, and then run a bunch of automated tests before going to prod. But let’s keep simple for now.

There are a few CI sites and tools out there supporting Dart natively, such as drone.io. The way they typically work is: check-in triggers a build, which also runs a bunch of tests, and a successful build triggers a deployment.

To get you started, here’s a simple CI setup for your Dart server which won’t require any of these tools yet – focus on your product and deal with making that technology decision later.

First, create a build&deployment script. Just a simple shell script will do the job for now, something like:

#!/bin/bash
rm -rf social
git clone git://github.com/MyUserName/social.git
stop social
rm -rf /opt/social/*
cp -r social/* /opt/social/
cd /opt/social/
pub get
start social

Second, set up a web hook endpoint on your production server, listening on some obscure port and triggering the deployment script on every POST. Of course, the web hook endpoint is also written in Dart:

import 'dart:io';
import 'dart:convert';
void main() {
   HttpServer.bind(InternetAddress.ANY_IP_V4, 8977)
     .then((HttpServer server) {
       server.listen((HttpRequest request) {
         if (request.method=="POST") {
           print('Webhook triggered at ' + new DateTime.now().toString());
           request.response.statusCode = 200;
           Process.start('deploy.sh', []).then((process) {
             process.stdout.transform(new Utf8Decoder())
                        .transform(new LineSplitter())
                        .listen((String line) => request.response.writeln(line));
             process.exitCode.then((exitCode) {
               request.response.writeln('Exit code = $exitCode');
               request.response.close();
             });
           });
         }
 });});}

Latest version of webhook.dart is here.

Lastly – add the web hook to your repository and make sure it gets triggered on every check-in. For example, in GitHub:

webhook

Nothing fancy – no security, no notifications, no build dependencies etc, but took only a few minutes to set up. Now write some server-side Dart code and push it out!

See also dart-upstart on GitHub, and feel free to discuss on HN.

Why Dart should learn JSON while it’s still young

2/14/2014 UPDATE 2: filed an issue for the DateTime->ISO8601 serialization. 

2/2/2014 see UPDATE at the end

Remember how easy it was to learn your native language as a toddler? Of course you don’t, and that’s the point. Grammar and vocabulary somehow got hard-wired in your brain while you were very young, and pretty much all you had to do was listen and sleep. As we all know, it’s much harder to learn another language later in life. 

One of Google’s youngest babies, the Dart programming language, must have been conceived some time in 2011 and was finally born in November 2013. So, it’s fair to say that it’s still it its infancy.

I like that Dart allows for using the same language and framework for browser development (potentially an SPA powered by AngularDart) and for back-end code. Yes, just like node.js. But without having to deal with Javascript.

Here’s one suggestion though I wanna make to Google’s Dart team: teach your baby Dart to speak JSON, the web’s data-interchange format, while it’s still young. 

“Wait a minute”, you might say, “what hell are you talking about? Dart does already speak JSON. There’s dart:convert, which allows you to decode and encode all day long.”

Yes. But for a language designed as a new platform for scalable web app engineering, JSON shouldn’t come as an afterthought, hidden somewhere in a library. JSON is everywhere on the web, and Dart should speak to you and listen to you in JSON like it’s its native language. The Dart editor should be able to display a JSON serialization of anything the mouse pointer touches. Dart should breathe JSON. Let me give you an example:

A few weeks ago, I found it surprisingly hard to serialize a simple (plain-old) Dart object like

class Customer
{
  int Id;
  String Name;
}

to this (very common) JSON representation

{
    "Id": 17,
    "Name": "John"
}

and ended up posting this question on Stackoverflow. It took me a bit by surprise when Google’s Developer Advocate for Dart, Seth Ladd, replied that

Unfortunately, there’s no universal JSON serialization of objects for all platforms.

Well, that’s true. JSON has been designed as a data interchange format, which doesn’t cover the platform and language specific serialization aspects. In fact, JSON is very lightweight by design. Side-note: I recommend reading RESTful Web APIs to learn more about emerging standards sitting on top of JSON, such as Collection+JSON

However, as said: the web speaks JSON. Almost every RESTful API speaks JSON (well, a few still speak XML). The Twitter, Facebook, and Stack Exchange APIs speak JSON. Therefore, I think the simple serialization format above – even if not part of any formal standard – should become the default serialization of a Dart object.

So, just typing

var json = JSON.encode(customer);

should give me the the simple JSON representation of customer by default. As pointed out in the Stackoverflow question, I should not be required to explicitly implement toJson() or use a third-party library such as Alexander Tkachev exportable package (although I have to admit it’s pretty neat).

There’s one more thing – DateTime. Unfortunately, ECMA-404 doesn’t standardize how to serialize dates. ISO 8601, however, does. Dart’s DateTime class already “complies with a subset of ISO 8601″ via its parse method..

So, here’s another thing I wanna add to my suggestion: consider adapting ISO 8601 for a default JSON serialization of DateTime, so the following just works:

var dt = DateTime.parse("2014-01-26T11:38:17");
print(JSON.encode(dt));

As of Dart 1.1.1, this will throw:

Unhandled exception:
Converting object to an encodable object failed.
#0 _JsonStringifier.stringifyValue (dart:convert/json.dart:416)
#1 _JsonStringifier.stringify (dart:convert/json.dart:336)
#2 JsonEncoder.convert (dart:convert/json.dart:177)
#3 JsonCodec.encode (dart:convert/json.dart:106)

Discuss on HN.

UPDATE - lots of very good feedback on Hacker News, here are some key points and my thoughts:

  • Symbols (see skybrian‘s top rated comment; also pointed out by floitsch): the kind of default serialization I’m proposing requires preservation of symbols, which presents challenges around minimization (keep in mind that Dart code has to run in a browser), inheritance, and dynamic types. Great point, I haven’t thought of that. A required class annotation such as @json (see zubspace‘s suggestion) might at least partially address these issues.
    By the way: some other JSON serializers such as ServiceStack.Text (.NET) use reflection to serialize all member fields of an object unless the class is decorated with [DataContract], [DataMember] attributes in which case serialization becomes opt-in per member field. Dart could at least mimic the latter behavior.
  • “JSON has no support for encoding new types into the serialization” (deathanatos‘ comment) – as pcwalton points out, this can be addressed by using Dart’s (optional) static type information. DateTime is a very common type and should have a default serialization, and it should be ISO 8601. To be exact, RFC 3339 (which is a subset), as daurminator points out.
  • “Can already be done” – for instance, floitsch points out that “It’s extremely easy to get the behavior you want. [...] This way you pay the price when you want to, and don’t force it on all users.” – I certainly agree with the latter point if adding a default/out of the box JSON serialization does indeed add significant overhead, impacting all users including those which don’t require it. On the flipside, if a cost-effective solution can be found – e.g. by providing a default JSON serialization via class annotations – I feel that it should be part of Dart’s core and not up to individual developers to home-brew their own solution to a very common problem. In the particular case of DateTime, I’m pretty sure that adding a default JSON serialization now will increase interoperability between Dart-based services and sites in the future.

If you love writing code, do not become a “consultant”. Become a developer.

^ This is my simple career advice to everyone who loves coding.

10 years ago, I was about to finish university. I’ve always loved writing code, but by then, I had the following misconceptions about being a software developer:

1. Writing software is just a lower, entry-level job.

2. There is not a lot of career advancement for software developers. To advance your career, you have to ultimately become a manager.

3. If you’re still “just” a software developer when you’re 40, you might be out of a job soon.

All three are wrong. Being a software developer is an amazing and rewarding career. If you keep your skills up to date and your passion alive, I guarantee you’ll still
be in high demand by the time you reach retirement age. And chances are, you might not event want to retire but continue writing code. It’s just so much fun.

My initial career choice out of university was to become a consultant. First, I worked as a management consultant for McKinsey&Company, probably the most glamorous
name in the industry. I spent 14-16 hours/day (yepp, really) at work, wearing a suit, sitting in meetings, digging through large spreadsheets, and preparing Powerpoint
slides. It was actually quite interesting at first: for instance, I worked on a large-scale ERP migration for an insurance company. There was one big problem though -
I didn’t get to write any of the actual migration code.

In my next job, I worked for a technology consulting company. Their projects were about actually building things and writing code. Great! Well, I still had to wear a suit (mostly) and sit in meetings, and occasionally write status reports. But at least some portion of the job was really about firing up my IDE and writing and debugging actual code. The job was well-paid and the hours still occasionally were long, but mostly reasonable.

There was still a big downside though: quality didn’t matter that much. Sure, the customer expected me to write good code and to meet deadlines. But really, there was
such a thing as good enough. The company I worked for didn’t care if my code was well-designed and maintainable. They cared about revenue. And as long as the customer
was reasonably happy and would continue signing contracts, they were happy (and generiously return a portion of their cut as an annual performance bonus). Actually,
writing high-maintenance code would be to some extent be a good thing since those additional maintenance hours of course were billable, too.

If quality doesn’t matter much, though, you’re not gonna get a lot better over time. You’ll end up doing the same mediocre things over and over again, working with mediocre people. That’s a bad thing. Don’t stay in an environment like that.

There was another problem: the only career advancement was to become a manager. So, I eventually became one. Only to (again) realize that I didn’t like it as much as
writing code: I spent time in meetings, strategy discussions, sales presentations – boooooring.

Finally, after a few more years, I made my best career choice ever and became a full-time developer. I spent a few years at Microsoft as a “Software Development
Engineer” (SDE), then joined Stack Exchange a year ago. Having fun and writing and shipping code every day. Oh, and I haven’t had to put on a suit to work for years.

If you’re truly passionate about writing code, don’t become a “consultant”. Find a great company (for instance, on Stack Overflow Careers) and become a developer!

Discuss on Hacker News.

How good is your team? Break production and you’ll know!

amazon down

Earlier this year. Somebody is having regrets right now…

If you’re a developer on a team running an online-service, the day will come where you break production. Not just the build. Not some automated tests. Not the development or staging environment. You will break the real thing that’s out there and used by customers.

Why? Because no matter how many layers of safeguards and process you apply, no matter how thorough your QA team (if existent) works – you still can’t predict the future. You’ll never know the exact sequence of bits flowing from clients to servers, from datastores to business tiers to front-ends.

Move fast and break things – not only at Facebook

At Stack Exchange, we rely on a variety of safeguards to keep the Careers 2.0 site up and running, such as unit tests, integration tests, UI tests and perceptual diffs. Shipping a change to production is fully automated, but it requires all these tests to pass first. So, as a developer I can ship quickly and as often as I want (which is fun) while still being protected to some degree from destroying our live site with the click of a button.

readyToShip

All tests have passed? Clicking the last “build” button will deploy to production. Note: no sign-off meetings needed.

Still, a few months back, I managed to break production. Here’s how: I checked in a database script which worked locally and on our dev environment, but then threw an error in production due to a difference in data. As said: atoms – universe – not can predict..

By the way – even if a production build fails, it doesn’t necessarily mean that the entire site is broken from a user’s perspective. Let’s say we’re currently running version n and kick off the build to deploy version n+1. In case of a deployment failure, we would ideally expect a clean rollback to version n. Well, what happened in this particular case was: my change got bundled up with a few others, includig a set of database migrations. And of course my db migrations failed after previous migrations, authored by someone else on my team, have already succeded. So, the database was basically now somwhere in state “n+0.5″, while the web tier was still running version n. This inconsistency turned out to be fatal.

We need to talk

The day you break production might be a stressful one for you personally. But the good news is, It’s also a day you’ll to find out what kind of team you’re on. If you get the opportunity to fix the problem, apologize for your mistake, and (ideally) take some steps to prevent the same type of failure from happening again, you’re on a good team. Someone might still yell at you in the heat of the moment. Don’t take it personally – the consequences are potentially huge if your site is down. Your company might be losing money with every minute of downtime. News about your failure might already be spreading on social media. Pressure is on.

Quite different story though if you find yourself reporting to eight different bosses after the incident, with fingers pointed at you all over the place. In case you’re part of a post-mortem discussion, pay close attention: those can be incredibly useful and constructive, and also a gigantic waste of time.

If you’re on that last team, run. Not only because chances are you didn’t enjoy getting yelled at that much. But much more importantly, in the long run you’ll find the entire team exercising risk aversion. If everyone’s scared to death of getting hassled (or fired) after breaking something, they will try to make sure that won’t happen – but at a high cost: be prepared for long sign-off meetings, senseless CYA email threads, and the blame game to already start before a single bit has been shipped. In extreme cases, it’ll take anywhere between forever and eternity to get anything done.

Well, now that being said…

Of course, you can’t be expect your team to be very forgiving if you do any of the following:

  • deliberately circumvent existing rules and procedures
  • try to cover up your mistake
  • make the same mistake several times
  • refuse to take accountability
  • generally signal that you don’t care that much

Ever worked with people who do that? It’s basically the dark side of “you should be allowed to break stuff”.  If you find yourself in an environment where this kind of behavior is tolerated, you also better run fast – before assimilation sets in.

Won’t happen again!

In the case of my failed database migration, I ended up improving our internal migrator tool, so all database migrations are now by default wrapped in a global transaction. No failed migration scripts have brought our site down since then – yay!

So, how good is your team? If you don’t know now, don’t worry. You’ll find out.

Shipping five times a day

Shipping code, I’d say, is one of the most satisfying aspects of being a developer. Shipping is fun! Your code is running in production. Customers use it. They might even be happy about it and tell you. Or, things might not work the way you expected and you’ll hear about that as well (and learn from it and get better). Either way, it can make you feel great and give you this warm fuzzy feeling of meaningfulness. Think Maslow’s hierarchy of needs for developers:

Maslow's_Hierarchy_of_Needs.svg

So, one of the things I really like about my current developer job at Stack Exchange is the fact that, thanks to continuous integration and deployment, I can ship code to production basically as often as I want. Occasionally, that’s as often as five times a day. That would be the case when it’s my turn to fix bugs reported by users on meta.stackoverflow.com or by our paying customers. 

Of course, getting brand-new features out the door often takes longer. Getting company pages right took a few months and several iterations, for instance. That doesn’t mean code just sits there hidden somewhere in branches and subbranches or even on developer machines. It’s still getting checked in all the time, the new stuff is just hidden behind feature flags. In-progress features light up on our internal dev environment. Heck, I’m still gonna call that shipping for now in Maslow’s sense. I’ve shipped my code somewhere, even if just to a small internal audience for now. Still fun. I’m happy. The real shipping – removing the feature flags, announcing to customers etc. – happens later and is even more fun, of course.

Now, throughout my developer career, I’ve worked in several larger organizations where this kind of instant-gratification, developer and customer happiness producing way of getting code out to production seemed impossible. Shipping typically meant all or a subset of:

  • Painful end-of-milestone branch integrations with merge conflicts all over the place.
  • Testing and re-testing. Does everything still work after all those merges?
  • Long meetings with many stakeholders to discuss “the deployment”, schedules, dependencies, downtimes…
  • Follow-up emails to those meetings. Follow-up emails to those follow-up emails.
  • Multiple hand-offs (give binaries to QA team, they will then give them to the operations team…)
  • Dealing with bureaucracy: deployment documents with step-by-step instructions. Sign-off meetings for those documents. Deployment checklists.
  • More discussions on “what we’ll do if something goes wrong”. Typical outcome: we need rollback instructions! In a separate document!
  • Long and uncertain working hours at “deployment time”.
  • And the worst of all: your ready-to-ship feature is getting “postponed” (that means “cut”) and doesn’t ship at all.

You’ve already guessed it – shipping code this way can be anywhere from tolerable to simply awful. If you’re a manager of developers on such a project, better up their morale with a big end-of-milestone “shipping party” – that’ll at least address some of their needs in that orange area of the pyramid above.

My point? If you’re a developer and about to join a new team, talk to them about shipping before you sign.

(Next post coming up: tools & culture aspect of continuous deployment, and how I broke something and didn’t get executed)

Plain Old Dart Objects

As a C# developer, I use auto-implemented properties all the time, typically to create POCOs:

class Customer
{
  public string FirstName { get; set; }
  public string LastName { get; set; }
  public string City { get; set; }
  public DateTime Created { get; set; }
}

The compiler will generate an invisible (well, sort of) private member field for each property, which can only be accessed through the getters and setters. Why is it a good practice not just make the member field public, by the way? Because we might wanna add some validation logic later, and changing it from a member field to a property will require recompilation on the caller side. Which is in particular inconvenient if the binaries have been shipped. Also, subclasses can override properties and add logic to them – in C#, it’s not possible to override a member field with a property.

Now, let’s write the POCO in Dart (which I guess makes it a PODO):

class Customer
{
  String FirstName;
  String LastName;
  String City;
  DateTime Created;
}

As simple as it gets. One thing you’ll notice is that there is no public access modifier. That’s because in Dart, there are no access modifiers: if a member attribute starts with an underscore, it’s private (to its library). Otherwise, it’s public. I think that’s great! Typically, programmers use the underscore notation (or a similar prefix such as m_) by convention anyways. So, having an access modified and the prefix is kind of redundant.

Also, unlike in the C# version, nothing appears to tell the compiler to wrap those attributes in auto-generated getters and setters. That’s because the Dart compiler always implicitly generates them (except for setters on final attributes)! Well, as an optimization, it might not really do that although the Dart documentation does make that claim. But the important part is that you can pretend that it does, and if you should choose to make the getters/setters explicit later (or override them in a subclass), you have a guarantee that nothing will break. For instance, here’s how we might add some validation logic later and/or in a subclass:

 String _firstName;
 String get FirstName 
 {
   return _firstName;
 }

 set FirstName(String value)
 {
   if (value == "Dirk") throw new Exception("No Dirks allowed here.");
   _firstName = value; 
 }

There’s one more thing: frequently, you’re going to instantiate all the member fields of a POxO at instantiation. Here’s how Dart allows you to define a constructor for just that:

Customer(this.FirstName, this.LastName, this.City, this.Created);

Hmm… syntactic sugar, sweet!

Good stuff from Dart

Google’s Dart language and SDK appear to be approaching a 1.0 release some time this year. Having spent the last few years mostly in C# and .NET land (still using it every day – I work on Stack Overflow Careers 2.0 which is built on ASP.NET MVC), I have to admit that I’m intrigued by Dart’s bold core premise: let’s not fix, but make a clean cut with JavaScript, the web’s lingua franca and its flaws. Instead, behold – we’re introducing Dart, a new class-based and (optionally) typed language. We’ll get all the OOP goodness back (real, not prototype-based inheritance, interfaces, generics…), and it’ll run both in the browser and on the server! Plus, an async programming model and no shared-state concurrency via isolates. Sounds good in theory!

Well, it won’t be that easy. Firstly, Google is pulling a VBScript by creating their own standard and incorporating it into their own browser. Other browsers aren’t likely to include a Dart VM any time soon and will therefore require compiled versions of all client-side scripting code. I bet at least minor inconsistencies between the the native and compiled versions will occasionally appear. And they will be hard to troubleshoot – as a developer, let me put on my sceptical face for a second here :/.

Also, there’s the ecosystem. It’ll take a while to catch up with the gazillions of JS libraries out there. Granted, it comes with more batteries included (no need for jQuery for instance), but still.

So, it’s hard to say at this time whether Dart will get widespread adoption or just disappear again within a couple of years. Nevertheless, I’m having a lot of fun studying the language and experimenting with it (oh yeah, that was your idea, Demis- thanks mate!). So occasionally, I’m going to blog about concepts and features in Dart which I like in particular. Who knows, maybe some of them will get ported to C# and other languages.

For today, let’s start by talking about something simple yet very useful:

Named constructors

Method names should be self-explanatory and not require additional documentation. In fact, often times adding a comment can and should be avoided by refactoring code segments into well-named helper methods. Unfortunately, constructors in C# can’t have names. In Dart, they can:

class Customer
{
  Customer.fromName(String name) { /* ... */ }
  Customer.fromJson(String json) { /* ... */ }
  Customer.fromXml(String xml) { /* ... */ }
}

To be fair: unlike Dart, C# supports method overloading and therefore constructors can typically be distinguished by their signatures. However, you’ll have to become creative if the signatures end up identical, like in the example above. Static creator methods (stuff like DateTime.FromFileTimeUtc) can be used to work around this problem, but named constructors are nicer and I think they would make sense in C# and the CLR as well.

new Blog()

So, I’m starting a new blog. Specifically, another coding blog. My previous personal blog has been quiet for a while and also wasn’t very focused.  It’s time for a clean start. Expect me to write about

  • things I’m working on at Stack Exchange, in particular Stack Overflow Careers 2.0
  • my $0.02 on languages and technologies – spoiler alert: I’ll start with Dart.
  • technical interviews and recruiting (we’re hiring at Stack Exchange, by the way)
  • software developer careers and life

Thanks for stopping by and stay tuned!