Simple service discovery using AWS Private Hosted Zones

A rather simple, but effective and easy-to-setup service discovery (SD) mechanism with near-zero maintenance costs can be build by utilizing the AWS Private Hosted Zone (PHZ) feature. PHZs allows you to connect a Route53 Hosted Zone to a VPC, which in turn means that DNS records in that zone are only visible to attached VPCs.

 

Before digging deeper in the topic, let’s try to find a definition for 'simple service discovery'. I'd say in 99% of the cases service discovery is something like "I am an application called myapp, please give me (for example) my database and cache endpoints, and service Y which I rely on", so the service consumer and service announcer need to speak a common language, and we need no manual human interaction. This is at least how Wikipedia defines service discovery protocols:

 

Service discovery protocols (SDP) are network protocols which allow automatic detection of devices and services offered by these devices on a computer network. Service discovery requires a common language to allow software agents to make use of one another's services without the need for continuous user intervention.

 

So back to the topic. You might think: Why not use Consul, Etcd, SkyDNS etcpp? 

 

no software is better than no softwarertomayko

 

You are not done with installing the software. You might need to package, configure, monitor, upgrade and sometimes deeply understand and debug it as well. I for one just simply love it when my service providers are doing this for me (and Route53 has actually a very good uptime SLA, beat that!) and I can concentrate on adding value for my customers.

 

However, service discovery apps introduce more complexity, magic, and point of failures, so don't use them unless you absolutely need to.

 

This is another point. Keeping it simple is hard and an art. I learned the hard way that I should try to avoid more complex tools and processes as long as possible. Once you introduced complexity it’s hard to remove it again because you or other people might have built even more complex stuff upon it.

 

Ok, we are almost done with my ‘Total cost of ownership’ preaching. Another aspect for me of keeping it simple and lean is to use as much infrastructure as possible from my IaaS provider. For example databases (RDS), caches (ElastiCache), Queues and storage (e.g. S3). Those services usually don’t have a native interface to announce their services to Consul, Etcd etc. so one would need to write some glue which takes events from your IaaS provider, filters and then announces changes to the SD cluster.1

Ok, so how can we achieve a service discovery with the AWS building blocks and especially Private Hosted Zones?

read more 0 Comments

The Jimdo Burnout Game

Rules

  1. When you come to work, pay $1 for each person of your team who is already there 
  2. When you leave work, pay $1 for each person of your team who is still there
  3. First one with a diagnosed burnout wins!
  4. If you know the rules, you're part of the game

 

Optional rules

  • Once a month, buy blow for the whole team
  • Once a year, get a psychologist for the winner
read more

Fast feedback is everything

As software developers, we spend a lot of our time writing code. Whether we're implementing new features, fixing nasty bugs, or doing boring maintenance work, there's always some code we either create from scratch or try to modify for the better. When writing code, we need confidence in what we do. We need to know whether our changes work as intended. That's why we write tests after or, preferably, before touching any code. So, inevitably, a huge part of our daily work comes down to these two things: programming and testing.


Programming and testing are, of course, only vague terms, and their meaning depends on how exactly you approach software development. For example, when practicing test-driven development (TDD), you typically follow these steps:

  1. Write a test that defines the desired behavior.
  2. Run the test to see if it fails.
  3. Write some code to make the test pass.
  4. Run the test to see that it passes.
  5. Refactor the code (and re-run the test).

Even if you don't practice TDD (you totally should!), you're probably still going to:

  1. Write some code.
  2. Write a test after the fact.
  3. Run the test to see if it's successful.

In order to write an actual program, you have to run through these steps -- this loop -- again and again. The point is that no matter what development method you prefer, constantly jumping back and forth between programming and testing comes at a price: it doesn't just slow you down in terms of time spent (especially with long-running tests), the involved context switching also drains your mental energy, which might ultimately destroy your productivity. For this reason I believe that the following statement is so important -- and I'm not tired of repeating it whenever I get the chance:


When it comes to programming and testing, fast feedback is everything.


Here, fast feedback means that the time between changing code (or tests) and getting results from running the tests is reduced to a minimum. In other words, you end up with a fast edit-compile-test loop (the compile step is optional, of course). The great thing about this, as Joel Spolsky put it: "the faster the Edit-Compile-Test loop, the more productive you will be".


There are a couple things you can do to shorten the feedback loop. Certainly the first technique that comes to mind is isolated testing, which involves eliminating (slow) external dependencies like databases. This topic has received a lot of attention lately and I won't go into it here (by the way, TDD isn't dead). You should absolutely invest in your tests and make them faster. Besides trying to implement faster tests, however, you can also optimize the way you run them.

 

Running tests the fast way

read more 10 Comments

The Jimdo Sessions Episode 11 - AngularJS with Robin Böhm

Time for another episode of The Jimdo Sessions. A few weeks ago, Robin Böhm visited us in our office in Hamburg. He's well known for having written the first German AngularJS book, which is already successful. At this point, we'd also like to thank him for sending us a copy. It's a great introduction to AngularJS, and if you prefer German books over English ones or you are just curious about what Robin has to share, make sure to check it out.

But enough of this, so we invited him to our office to share his thoughts about the expert levels of AngularJS. Because at Jimdo, we already use AngularJS in the latest parts (we will share details when we have released that thingy). Of course, we also recorded his talk, so you can also enjoy it now. So here you go:

read more 5 Comments