Functional programming in the enterprise world

Recently I have been heavily using Apache Spark. For those of you who don’t know Spark is a very powerful system for working with data and parallel that is written in the Scala language. Scala is not new, but certainly on the “newer” end of the spectrum. Today new languages are coming out all the time so 12 years is fairly long. What many people find attractive about Scala, at least I do is the fact that it runs on the popular JVM. In fact a developer may be able to write code in Java and interact with code that was written in Scala. The challenge is striking the balance between closeness to Java and still providing whatever it is that the creators of the language hope to achieve.

I am a big believer in Object Oriented software design and development. I’m not saying that every project in the world needs to be written exactly the same, as they all have different requirements. I will say that for enterprise software the level of adaptability is truly best achieved with levels of modularity as well as abstraction. In truth, if you can achieve the principles that an enterprise architect looks at in a technology it may be something to be considered. In the past languages like LISP such as Haskell, were never geared towards the enterprise as their mathematical background and dynamic typing didn’t fit the bill for the type of type safety compilation and code reuse that has been found in other enterprise technologies. In general I like languages that aim for simplicity but at the same time aren’t overly opinonated. I recently read a rant about the Go language for the lack of support of assertions because the language creators felt that people used assertions incorrectly, not like Java chose to avoid pointers because of their inherit nature to cause errors. This is an example of a language being dumbed down or muted, with that said I really like many aspects of Go.

I’ve read a number of articles about so called “veteran” developers who have ditched OOP to embrace some sort of functional language. Complaining that the design principles of OOP aren’t applicable and don’t work. I even read recently that a college professor from Carnegie Mellon removed OOP from the syllabus for freshman. I don’t necessarily think that is too awful, but I do think problem decomposition that one would do when designing an OOP system is not only very helpful but also useful. Not everything easily fits into a “map” or a “reduce”. I can’t speak for all people, but I think that OOP is more natural to the domain than functional. If you truly understand the domain and how to break apart a problem into single scoped entities you will find simplicity and elegance.

With that being said, I think that for parallelism functional programming has always been faster and more efficient. I do however think that there is room to bridge the gap. Scala is a multi-paradigm language, not just functional. I believe it is that aspect that can truly bring something special to the table. Technologies like Spark still require a more or less functional approach. There are layers like Dataframes and graphs that attempt to abstract some of the functional aspects from the developer. What I haven’t yet seen is what Hibernate and other ORM technologies did for SQL, with respect to large scale functional parallelism. I think once we bridge that gap that will be the holy grail for enterprise software. I look forward to seeing how these technologies continue to evolve and mature.