I recently gave a talk about NodeJS at the JAX conference in Santa Clara. Here are the abstract and slides.

JavaScript used to be confined to the browser. But these days, it’s becoming increasingly popular in server-side applications in the form of Node.js. Node.js provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use Node.js effectively in modern, polyglot applications.

And, here is the video

I also gave my Developing with cloud services talk.

I also enjoyed John Kodumal’s Scala Typeclassopedia talk on Scala type classes. A great explanation of some concepts that are covered well.

I spent tuesday at DeployCon, a conference about enterprise platform services. Some of the sessions were great. Most notably,  Dave McCrory’s Data Gravity talk, Das Kamhout’s talk about Intel’s PaaS journey, and David Mortman Cover your PaaS talk about PaaS security. Here are a few tweets from the conference.

Those talks were great but there were also several panel sessions that were, to put it politely, very unsatisfying. This problem is nothing specific to DeployCon. Most generic (not focussed on a particular product/service) cloud events that I’ve attended over the past five years have been equally unsatisfying. While the “What is a cloud?” discussion that was prevalent in 2008-2009 has mostly gone away, so much of what is said during the panel discussion consists of vague, high-level generalities. Periodically, I wanted to shout: “I don’t know what you are saying“.

So why is this? Recently I’ve been reading the excellent book To Save Everything, Click Here: The Folly of Technological Solutionism. One interesting point in the book is that “the Internet” is a vague, nebulous concept and that when we talk about it rather than the specific inventions, people, and companies that are utilizing the network, “our technological debates will remain lazy, shallow, and unproductive”.

Perhaps the same is true when it comes to “Cloud” and “PaaS”. Those terms are simply too vague and nebulous and that if you want to have a meaningful discussion then you need to talk about specific products, people and companies in that space. So, for example, if you are organizing a Cloud/PaaS conference then have real users talk about their experiences deploying applications with a specific public PaaS; or describe how they built a private PaaS. Have users describe their failures with PaaS. And instead of talking about PaaS and big data, what about their experiences with NoSQL database X and PaaS Y. Instead of lots of handwaving fill the schedule with one concrete example after another.

I’ve spent the past 9 weeks (i.e. weekends) taking Caltech’s Learning from Data online class. Each week I watched two hours of lectures and did the homework, which was a multi-choice quiz that typically required writing some machine learning code. Overall, it was a great experience and I’m glad that I took the class.

I learned a lot about machine learning algorithms; got to exercise my very rusty math skills since a lot of machine learning involves statistics, calculus, and linear algebra; had a hilarious time trying to explain a gradient descent algorithm to my family; and had the experience of having 3 hours before the homework deadline to  learn enough Python (a language I’ve avoided until now) in order to use a nifty quadratic programming package.

This was my first online class. I’ve come to realize that despite all the hype, online classes have a mix of benefits and drawbacks. On the one hand, I liked the convenience and low cost (ie. free) of the online class. It’s unlikely that I would have found the time to do a real world class. I also learned a lot from the class so that was definitely a good thing.

On the other hand, the format of the class could have been better. Each week, I had one shot to take the test. This mean that sometimes I lost 30% because of some stupid error:

“what you mean excel’s log() function is base 10, even though there is a log10() function too!”

“when computing support vectors, anything < 1e-05 should be considered zero. Seriously?!”

“So the identity matrix only has 1’s on the diagonal. Doh” (Told you my math skills were rusty 🙂 )(Note to prospective employers: this was feedback from a friend, not from me 🙂 )

It would have been less frustrating to have had either test data to validate the code or multiple quiz’s.

It also would have helped if I’d had a real human being to talk to about. You know, like in those totally obsolete, bricks and mortar colleges. The class does have a forum, which was helpful, but 1-on-1 time with a tutor is priceless. Coincidentally, I’m currently reading  To Save Everything, Click Here: The Folly of Technological Solutionism, which is an incredible book. It references a study that found that the best predictor of student success was the amount of interaction they had with professors. Of course, that costs money and doesn’t scale to 50K person-sized classes, which I think the online class fans will need to accept at some point.

So ironically, there were some real challenges for human learning in the machine learning class. But despite those challenges I’m glad that I did it and would highly recommend the class. Another session is starting in April.

Last night I went to really interesting Scala BASE meetup at LinkedIn. The topic was Scalable and Flexible Machine Learning With Scala (slides). The large room was packed with over 300 people! I was sitting down the front, which made it difficult to see the audience, but my impression of the polling was: most folks were developing in Java, many were using Scala and lots of people were doing  machine learning.

The speakers were Chris Severs from eBay and Vitaly Gordon from LinkedIn. The talk started with a characterization of the different types data scientists and the pros/cons of the way they write map/reduce jobs:

• Mixer – uses what ever combination of tools work such a combination of Apache Pig and Python: a python script that executes an Apache Pig script that invokes a user-defined function written in Python.
• Single language expert – uses the one tool that they are familiar with, e.g. Perl scripts +  Hadoop streaming
• Craftsmen – write Map/Reduce jobs in Java and specify low-level configuration details.

Next, the speakers claimed that we need a better, more pragmatic approach: “A five tool tool” (see Five Tool player): agile, productive, correct, scalable, simple. And such a tool is Scalding, which is a Scala-based DSL developed by Twitter for writing Map/Reduce jobs in familiar functional style. Looks impressive!

The final part of the talk consisted of some machine learning examples:

• Using a decision tree to estimate insurance risk of Titanic 2 passengers
• Using  streaming K-means clustering (coming in Mahout 0.8) to analyze eBay data
• Using the PageRank algorithm to analyze LinkedIn endorsements to rank Scala expertise in the bay area and amongst attendees of the meetup

During the Q&A there was an interesting discussion of Python vs. Scala for machine learning. Python has lots of mature libraries (Numpy, Scipy, …) but the claim was that

• many libraries are available in the Java/Scala ecosystem and that
• unlike Python Java/Scala can scale beyond a single machine by leveraging Hadoop. 

Seems plausible but anyone else care to comment?

I recently spoke at the excellent DevNexus 2013 conference, which took place in Atlanta,  Georgia. Below are my presentations. You can find example code here.

Developing polyglot persistence applications

NoSQL databases such as Redis, MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offer significantly better scalability and performance. However, using a NoSQL database means giving up the benefits of the relational model such as SQL, constraints and ACID transactions. For some applications, the solution is polyglot persistence: using SQL and NoSQL databases together.

In this talk, you will learn about the benefits and drawbacks of polyglot persistence and how to design applications that use this approach. We will explore the architecture and implementation of an example application that uses MySQL as the system of record and Redis as a very high-performance database that handles queries from the front-end. You will learn about mechanisms for maintaining consistency across the various databases.

Decomposing applications for scalability and deployability

Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.

In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.

Developing applications with Cloud Services

Cloud computing isn’t just about application deployment. There are also a growing number of cloud-based web services that you can use to develop your application. One of the most well known is Amazon’s Simple Storage Service. But there are many others including web services for messaging, relational and NoSQL databases, email and telephony. Using these services allows you to build highly scalable applications without the pain and cost of having to develop and operate your own infrastructure.

In this presentation, you will learn about the benefits and drawbacks of these Web services; their typical use cases and how to use them. We will describe a location aware, telephony application that is built using cloud services such as twilio.com and factual.com. You will learn about strategies for building resilient, fault tolerant applications that consume cloud services.

During my recent trip to Hyderabad for SpringOne, I had one day free for sightseeing. Using the google, I discovered what turned out to be an incredibly good tour guide company: DetoursIndia.com, which specializes in personalized, distinctive tours around Hyderabad. After talking to them I discovered that my free Friday, was the one day of the week when many of the tourist places are closed. However, Kavitha and Jonty at DetoursIndia.com worked hard to put together an interesting itinerary.

The day started with Kavitha picking me up at my hotel. We drove across town to the Moula Ali Dargah, which is very important Shia Muslim shrine that’s over 400 years old. Along the way, Kavitha, who is extremely knowledgable and a super fun and interesting person to hang out with, gave me a quick history lesson about Hyderabad and the surrounding area. The shrine, which is includes a mosque, is built on top of hill and has very impressive views of the city.

After getting our exercise walking up and down the many steps to the shrine, we then drove to the nearby memorial tomb of Mah Laqa Bai, who was an 18th century Urdu poet, and a powerful woman in the court of the second and third Nizams of Hyderabad. The tomb consists of some beautiful buildings and gardens. It is a very peaceful place with some interesting history.

The next stop on the tour was a cooking demonstration/class and lunch at Kavitha’s parent’s house! This was very cool. Her parents were super interesting. Kavitha’s mother cooked some delicious food and provided me with some great tips on preparing chapati and paratha.

After lunch we drove to a nearby vegetable market that I enjoyed walking around.

The penultimate stop was a quick walk around Charminar. A seriously intense, busy place with some beautiful buildings.

After visiting Charminar we then walked around Paigah Tombs. The tombs consist of impressively ornate buildings set in a very peaceful guarden.

Overall, it was a fantastic day.  If you are visiting Hyderabad I highly recommend that you talk to DetoursIndia.com.

A few weeks ago I received an email asking for the slides for one of my talks he had seen on parleys.com. What was especially interesting about this request is that it was for a talk I gave – what feels like a lifetime ago – at SpringOne 2007 !

The talk is Improving application design with a rich domain model and is about what is still one of my favorite topics: object-oriented design. It describes how to improve the design of your application’s business logic by using traditional object-oriented design techniques. It covers a number of topics including:

• the limitations of procedural code aka. Transaction Scripts
• the benefits of the Domain Model pattern
• the building blocks of Domain-Driven design  – entities, value objects, repositories, and services
• Common code smells and how to refactor them away: long method, feature envy, data class, primitive obsession, switch statements, data clumps

The talk finishes with a demo of how to incrementally refactor a procedural design into an object-oriented design.

These are some basic software development techniques that I suspect many developers have either forgotten or never learned in the mad rush to master the latest fashionable technology.

Fortunately, I was able to find the presentation on my hard-drive and have posted the slides with the embedded video of the demo to slideshare.net. You can also checkout the example code.