Tag Archives: machine learning

Mankind is not a simple feature vector

I’m not going to talk about politics, but I do want to talk about this article¬†I found on one of my many feeds. I will summarize this extremely briefly. This guy was so baffled by the election outcome and the poor pollster statistics and generally how this wasn’t predicted with machine learning and the sort.
Anyone who knows me, knows that i hate the term “Artificial Intelligence” I remember some expert in the industry who closer compared it to “slightly less dumb”. I have written a number of other articles expressing how we are very far away from anything that is truly intelligent, by much any definition. Tools Ike Watson and DeepMind are impressive highly refined tools that have been fed incredibly large data sets so that it could either be trained or train itself.
It is not at all a surprise that predictions were so far off. When dealing with decision making in general most people choose choice A because they align themselves with many of the principles, philosophies, and ideas. That is at least what we wish people did. The reality is the things that are not about the “issues” are a great deal more important than people may think. People are not Vulcans; they are not making every action or even most of the actions that they perform weighing in all of the possible outcomes and the long term versus the short term benefits.
People are emotional and erratic creatures. People are bigots, racists, or so open minded that they have no actual opinions. We live in a world where people must be politically correct and are shunned for deviation. Money and power go hand in hand provide ample leverage to many scenarios. Corruption is not Republican or Democratic phenomenon, it is a part of humanity. Our nation and its capitalistic mindset is one of the greatest things that make us a unique and great nation. Regulation from monopolies is a way to limit that the gross power a company may posses should only reach so far.
The power of leverage from a huge conglomerate had over the government is tremendous. A true system or checks and balances that provides independence to the capitalistic nature of commerce as well as the freedom for a nation to govern itself is not any easy task in any way. It seems evident that in our over 240 years it is still far from perfect.
What we need to realize is a machine model, and statistics that are collected only account for the features and dimensoins that were taken into account. The nuances of what a single person wants let alone a nation of millions of them is simply impossible without first grasping the human factor. The incredible thing that define each and every person who lives on this planet.
I hope that our understanding of ourselves as well as our capacity to teach our machine models improves so that we can better harness and sustain the wonderful qualities of mankind.

Data Evolution to Revolution

I apologize in advance as this is an attempt to solidify some ideas that I have been having into something a tad more cohesive. Let’s start small. Not sure if this is a chicken and egg sort of thing but I think it just may be. Does the evolution of technology yield more data and thus more complex and intricate mechanisms to capture, evaluate, utilize and wield said data are needed. Advances in technology bring about more data quicker and more accurately. Strides forward in technology enable greater insight into data that may have existed for years maybe even decades or more. Most of these technologies are merely building on top of their predecessor taking things one step further and refining them. Occasionally there are novel ideas that truly disrupt the pace and direction shattering preconceptions and misconstrued understandings. We have grown very little in our “PC” age in true advancement. We have sleeker, smaller, faster infrastructure. We have smart watches, phones, tablets and more. We have this buzz word “cloud” and “IoT” that people love to throw around. Anyone today can make an “app” and they think that is novel, that will change the world. We are so far away from true advancement on the computing level to use the word of intelligence. I would not pretend to be an expert on anything to do with AI or machine learning. I do however know that we have neither the precision or speed capable of coming remotely close to anything of substance. We are playing “Go” and Jeopardy, we are writing cookbooks, and more. True creativity is alien to our creations. We are doing nothing more then creating formidable copy-cats. Sure it may consume many different approaches to chess or some other topic. Ultimately it is illustrating a path that will attempt to “beat” its opponent. I am not enough of a philosopher or scientist to evaluate the state of that level of comprehension. It is certainly complex and well structured, and it may beat a human. It is however a very far cry from the human intellect.

Now that I have gotten this far I can try to say with a bit more clarity that I am trying to illustrate that computers and technology evolve constantly. Through their evolution they yield more and more data. New technologies are needed to understand and discover all of the depths of that data. Ultimately we are further burrowing into the initial data acquired. Nothing new only uncovered.

To teach a computer to act like a man you cannot, but you can empower a computer with the collected knowledge of a man and enable it to utilize that knowledge.

The inspiration for this rant is because I have been toying with the notion of creating a layer that will assist in abstracting data persistence from the application developer. This layer won’t understand your data only understand how it relates to each other, the types of data, and the types of analysis performed on that data.

The primary difference between TCP and UDP is fairly simple and can illustrate the beginning of this abstraction TCP has error checking and used when order matters. If you care about speed and nothing else, UDP is appropriate. TCP is heavy but reliable for data integrity, order, and consistency. I’m sure I’m not the first to draw a parallel to that of the traditional RDBMS your SQL variation versus the newer NoSQL variant. SQL is typically¬† characterized as transactional, a strict schema, relational and challenging to scale horizontally. Whereas the NoSQL variant is typically assumed to be non transactional, schemaless and easy to scale horizontally. There of course have been the newer variants often dubbed as NewSQL which exhibit a hybrid approach attempting to take th best qualities of the aforementioned technologies and provide a hell of a solution. There have been advancements in technologies specific for very large file store, JSON documents, and full text searchable data. Suffice to say, there is no size fits all solution. Depending on the software needs, business requirements among many other factors there may be dozens of different factors that can attribute to proposing a design. The DBA used to be the overlord with his stored procedures and locked down tables. Not every problem can be solved with a store procedure. With an ever growing technological world, and more and more data emerging a strategy to stay afloat ahead of the curve seems faint and distant. I think that with the right approach it can be done and change the nature of how we interact with data today.

Spring Data hopes to provide a layer of abstraction when accessing data from different sources. Varying from SQL sources, NoSQL variants, In-Memory sources or others. With Spring Data the @Entity represents your top-level data that we are dealing with. The other being the @Repository which is the abstraction layer that interacts directly with the store engine that is configured. Spring Data can support many different types of store engines but ultimately the onus lies upon the application architect to decide what the repository connects to. Image if there was a layer that determined how the defined data would be persisted evolving over time.

The relationship between an FPGA and processing is similar to this new layer and the data that it will persist. With an FPGA the gates are constructed according to the task at hand and how the module was programmed to react under those given circumstances. Similarly, this new layer which I am going to dub as an “Adaptable Data Source Gateway” will utilize the different components that are at its disposal based on the data inputted and the configured priorities.

Here is a high level overview of how this may be accomplished. The comparison to an FPGA only goes so far. An FPGA doesn’t “automatically” change its design, rather it changes its design based on its programming. To add this functionality with regards to data it will be necessary to maintain a list of possible storage engine types (engine interfaces) as well as implementations of those interfaces. We will also need rules to identify what data requirements are best suited for each interface. Ultimately we need a metadata system that will allow the developers to identify the data from afar allowing the system to gradually grasp the data needs.

I have some ideas for the secret ingredients that will make this possible. I am going to toy with these ideas so far and do some rapid prototyping. I hope to write more soon about my findings. As always any input is appreciated!