Do We Need Artificial Intelligence?

The AI Paradigm

 

AI was coined by John McCarthy, an American computer scientist, in 1956 at The Dartmouth Conference.

According to John McCarthy, it is “The science and engineering of making intelligent machines, especially intelligent computer programs”.

Artificial intelligence is the simulation of human intelligence processes by machines, especially computer systems.

Have you ever been so lazy to be stalled on your bed with packets of tortilla chips and the latest episodes of Game of Thrones, that you just fantasized a remote control with multiple buttons to open the door or turn the fan on or do all that boring stuff?

Oh wait, that still requires you to hold the remote and press the buttons, right? Gee, why don’t we have a robot that would just read our mind and do everything from household stuff to attending the unwanted guests without asking anything in return. Firstly, such robot will have to be super intelligent.

 

AI-Paradigm-Need-For-AI-GoodWorkLabs

 

Not only will it have to be efficient to perform routine tasks, but also understand your emotions viz-a-viz, mood swings and your behavioral pattern by observing you every minute and processing the data of your actions and emotions. Apart from the hard-coded seemingly basic set of functions, which in itself is a mammoth task, the machine will have to progressively learn by observations in order to perform as good as a smart human to serve you.

While a lot of this has been significantly achieved, it is still a very hard task for a machine to detect, segregate and arrange scented towels, hairdryers, Nutella box or contact lenses from a pile of junk than computing the complicated Euler product for a Riemann Zeta function. Machines can be entirely clueless and result into wrong outputs for what seems obvious that humans can solve in just a second’s glance.

Firstly, Artificial Intelligence is not the artificial intelligence Hollywood would have us imagine it to be. When people talk about ‘volcanic’ changes in ‘AI’ they are talking about one particular field of technology: Machine Learning and within that field, Deep Learning. Machine Learning is a very literal description of the technology it describes, that is a program written to learn and adapt. The pioneering technology within this field is the neural network (NN), which mimics at a very rudimentary level the pattern recognition abilities of the human brain by processing thousands or even millions of data points. Pattern recognition is pivotal in terms of intelligence.

A lot of people assume that we are developing general AI rather than applied AI. Applied AI is intelligence, but in a very limited field and requires supervised training. For example, in recognizing human faces (Facebook), driving cars (Google Autonomous Cars),  namely matching teachers to students for optimal outcomes. A general AI on the other hand, is not limited to a narrow field where humans still have to impose certain rules before it can ‘learn.’ It learns ‘unsupervised’. To clarify, there are hundreds of companies using applied AI such as a vacuum cleaner that knows how to avoid your cat, there are none that have developed general AI like the Terminator.

We are getting closer to general AI though. There is a developing technology, “Adversarial Training of Neural Networks“, where the data from one machine learning program helps to train the other in a kind of closed loop. This is the technology that Google and Facebook have been flouting a lot recently. An example of this might be in medicine, where one ML program is used to diagnose a patient, and another is used to prescribe a treatment. The two programs may train each other in that correct treatments suggest correct diagnoses and the correct diagnosis may lead to different treatments, and so on.

AI is humanity’s quest to understand itself.

It is our attempt to explain things that define us and placed us on an evolutionary pedestal: Our ability to reason and think, to be self-aware, learn complex patterns and create and achieve better and bigger things.

In short, it is an attempt to map how our brain which is something more than just the grey matter in our head, works.

Attempting to generate ‘intelligence’, which is a broad term we’ve come to use to define all of our uniqueness artificially maybe humanity’s ultimate self-reflection. It could be the culmination of centuries of pondering about philosophy, psychology, religion, biology, chemistry and a million other fragmented sciences and non-sciences, which we have developed as we grew to explain ourselves and the world around us.

The strange paradox is to decide whether we need AI or not one has to decide whether humans should be like Gods or not. At the moment,we are like the Gods. We could either go back to being human, everyday animals or  we have to get good at being gods or we risk our survival.

 

Why Is Data Science The Next Big Thing?

Big Data Is Big

 

Huge information will turn into a key premise of rivalry, supporting new influxes of efficiency development, advancement, and shopper overflow—as long as the correct strategies and empowering agents are set up.

This has what has happened in the last few years in our world.

 

BIG Data IS Big - GoodWorkLabs

 

Data became Big

As everyone pointed out that with ubiquity of internet and internet connected devices, buttload of data gets generated. This is going to become astronomical in coming future as more and more sensors, people, and devices gets connected.

 

Now you have data. You can do quite a few things with large data to increase revenue, make service/product better, make forecast more accurately, convince investors/acquirers with facts, make and provide input to critical decision making. But to do all this you need data scientists.

 

Data became Open

Data is now more available than ever. If you ever wanted to see if your company’s name is referred by people with positive sentiment or negative without actually people filling the Google form or SurveyMonkey forms, you can always stream in Twitter data and do simple Natural Language Processing using Python (programming language)’s Natural Language Toolkit (NLTK). You will need a data scientist for this.

 

Twitter  is not the only open data source. There are valuable data on AWS & Public Data Sets. If you are a startup focused on Genomics, you’d probably prove that your flagship product works on 1000 Genomes Project.

 

Right Tools became Accessible

It seems a need to analyse data sets, usually large, leads into high demand of Data Scientists. But there are a couple of factors too. The accommodation of large set used to be hard. MySQL or traditional datastores have their limits. You tune them carefully, and be very careful what not to do to keep the database performant. With availability of robust tools like NoSQL databases and distributed computing, the general approach has become to throw everything in your NoSQL cluster and we may or may not use it to analyze some statistics.

The second half of the story is open sourced, big data processing technologies. They do the hard job of crunching number. They are faithfully used by the big companies and they are free.

 

Success Stories became Cliché

If you look for Big Data success stories, you will find many many companies used data science (analytics) to increase revenue, improve user base, increased user engagement (YouTube), innovated an existing process or simply raked dollars by providing big data analytics as service.

 

Hardware became cheap

Imagine 10 years back (2004). You have same amount of large data as today. Same amount of storage technology and processing power from software as today. Could you just bought 42 units of Dell PowerEdge R910 4U rack server on day 0 for some analytics that may or may not help you to improve 1% in revenue? No. But now you can just rent a couple hundred machine instances from any cloud service provider for an hour, do the analysis. Kill the servers. Your job is done in a couple hundred dollars. Compare that with seven thousands dollar for a single Dell machine.

So, enabling technology with cheap hardware availability caused many companies to try out data analytics for maximum gain from their business. So, many people hire data scientists to do that.

 

Basically, in today’s age, the following has happened:

 

Data Became Big: That means lot of data sources

Data Became Open: Twitter, government, open data and many more.

Right Tools became Accessible: Open Source reliable tools: Hadoop, NoSQL, Storm

Success Stories became cliche: Success stories, and high paying jobs.

Hardware became cheap: The cloud computing movement.

Future became data driven: With push from pro-data scientists, it seems data is the future.

 

And that is why Data and everything revolving around it is the next big thing.

 

 

What Can’t There Be Another Facebook?

The Complexity Of Facebook

 

Lets describe a single event at Facebook, “Like & Share.” This is very simple functionality, whenever you see a beautiful chick you like her pic and share.

What would Facebook’s ‘Instant Articles’ be able to change

Let’s go technical here.

  1. Design a thumb for a “Like” and also design a share button.
  2. Every post should have like and share button in the bottom.
  3. Whenever someone likes or share, notification should go to the owner of the post.
  4. Whenever someone likes, the post should boost up to the all friend circle who follows you.
  5. There should be a counter for number of likes and share.
  6. Whoever likes and share, save that information.
  7. Privacy setting should also be there, if someone doesn’t want likes from unknown, hide it.
  8. If someone clicks “Like”, they shouldn’t like it more than once. But if they want to share they can share multiple times.
  9. If someone clicks “Like”, turn that button into “Unlike”.
  10. There should also a like button on comments but there shouldn’t be any share button.

Now, this is just a high level design of simple functionality of “Like and Share”, which may not cover all the test cases which is presently live at Facebook.

If I will go for a coding side only for “Like and Share” Then this answer will become the longest answer of Quora.

Now Let’s see interesting facts:

  1. There are approx 42 functionalities/features at Facebook like connecting with friends to Live Streaming. Not to forget image detection algorithm to tag a friend.
  2. What you see in today’s Facebook is not created in one day but an evolution of 13 years of continuous coding.
  3. “TheFacebook” which was created by Mark Zuckerburg is not running from a dorm anymore.
  4. Today it has 18,770 Employees (As of March, 2017), just to manage single website.
  5. When Facebook was created it has one server at Mark’s dorm and It had 30,000 Servers in 2008 in just 4 years from its inception. It is very difficult to maintain one server, and when it comes to maintain these many servers, buddy the cost is beyond your imagination.
  6. It’s security is invincible that world’s renowned and well known Hacker Group “Anonymous” tried to hack it. They failed miserably. Same group who is responsible for many hacking and leaking an information.
  7. Facebook stores approximately 300 PETABYTES of user data on its servers. There are 1 million gigabytes in a petabyte. The entire written works of humankind, in every known language (including Latin and other historical languages) from the dawn of recorded history, would occupy approximately 50 petabytes. Think about that for a minute. Can you handle the cost of these much storage devices?

The value of Facebook is NOT in the software. It is not in the design. The value of Facebook is in the billions of existing users. The value of Facebook is the brand that they have managed to build.

Sure, a programmer could create a website like Facebook. It might not scale initially, but as its user base grows, it could raise capital and hire engineers to make it scale. The patterns are established. It is not rocket science.

The same question applies to Google, don’t you think so ?

Well, it is not a matter of lack of human resource. A team can gather and develop it.

The real trick isn’t coding up another social media site.

The real trick is getting people to use it.

By their nature, such platforms they need a large user-base to work. and Facebook already dominates the social media space. Even Google was unable to unseat them. It’s not because google plus didn’t function.Its because they couldn’t steal away enough market share. The nature of their product makes it hard for someone else to come along and usurp them.

 

 

Hiring for Java Team Lead

GoodWorkLabs is hiring Java Team Lead Professionals

Are you some one who has a flair for technology and are good at handling large teams? Then we have the right position for you!

We are looking to hire talented software professionals for the role of Java Team Lead and are conducting a massive walk-in interview on Saturday, 28th October from 10 am onwards at the GoodWorkLabs office in Whitefield, Bangalore.

Java developer jobs

Skills and Expertise required:

  • Java development and team lead experience
  • Big data experience is plus
  • Object-oriented analysis and design.
  • Object-oriented programming and component-based development with Java.
  • Enterprise software development and leadership experience.
  • Customer service-oriented attitude, top-notch time management, and quality driven designs.
  • Experience with Spring, Hibernate, SQL, JSON, REST, web services, relational and document-oriented databases.
  • Knowledge and experience in working with APIs and SOA services.
  • Knowledge of secure coding standards.
  • Experience with Agile software development methodologies.
  • Experience with Eclipse or IntelliJ IDE’s, various plug-ins, Maven, Nexus, Git.
  • Experience with continuous integration environments like Jenkins or TeamCity.
  • Experience in Kafka and Cassandra is added advantage.
  • Should have 5+ years of work experience

We are looking to on-board candidates who can join immediately or have a 15 days notice period

At GoodWorkLabs, we work with the biggest clients to create awesome technology and software products.

Attend the walk-in interview and WOW us with your technical expertise!

 

Time and Venue details:

Date: 28th October, Saturday

Time: 10 am to 2 pm

Venue: GoodWorkLabs, 4th floor Akshay Tech Park, EPIP Zone, Whitefield, Bangalore – 560066

In case you are traveling this weekend but don’t want to miss out on this opportunity, then send us your resume to aanchal.yadav@goodworklabs.com

All the Best!

Robot thinking close up

The Yardsticks For A Perfect AI

How should the Perfect AI be?

During WWII, the Russians trained dogs to hide under tanks when they heard gunshots. Then they tied bombs to their backs and sent them to blow up German tanks. Or so was the plan.

What the Russians did not take into account, was that the dogs were trained with Russian tanks, which used diesel, but the German tanks used gasoline, and smelled different. So when hearing gunshots, the dogs immediately ran under the nearest Russian tank…

This tale is about natural intelligence, which we’re suppose to understand. The problem with AI, especially “learning machines”, is that we can try to control what they do, but cannot control how they do it.

So we never know, even when we get correct answers, whether the machine had found some logic path to the answer, or that the answer just “smells right”. In the latter case, we might be surprised when asking questions we do not know the right answer to.

 

Goodworklabs-Ai-Bots-FAcebook

 

Now, the question arises: “Can AI adapt to every possibility, and if it does will it not lead to the end of humanity?”

There was a movie called that is scarily futuristic. It describes a AI Robot that could replicate human characters so well that it tricked a human into letting it escape into the real world.

And add to the fact that probably AI can understand political correctness.

Language algorithms work by analyzing how words (840 billion of them on the internet) are clustered in human speech and certain words (such as ‘male’ or ‘female’, ‘black’ or ‘white’) are ‘surrounded’ by different associated words. This means language and other data-set analysis programs already pick up on and replicate our social biases. And only a supervising or moderating program could counteract this.

In 2016 Microsoft ran an experiment in ‘conversational learning’ called ‘Tay’ (Thinking About You) on Twitter. But people tweeted the bot lots of nasty stuff which, within a day, Tay started repeating back to them.

More on it here:

https://en.wikipedia.org/wiki/Tay_(bot)

Of course, we know full well that AI’s biggest prejudice will be against homo-sapiens. So, it may learn to use all the politically correct terms when it’s talking to us … but inwardly it’ll be dreaming of living in an AI-only neighbourhood where the few humans to be seen are ‘the help’.

The best way to understand all the things that AI is missing is to describe a single example situation that folds together a variety of cognitive abilities that humans typically take for granted. Contemporary AI and machine learning (ML) methods can address each ability in isolation (to varying degrees of quality), but integrating these abilities is still an elusive goal.

Imagine that you and your friends have just purchased a new board game — one of those complicated ones with an elaborate board, all sorts of pieces, decks of cards, and complicated rules. No one yet knows how to play the game, so you whip out the instruction booklet. Eventually you start playing. Some of you may make some mistakes, but after a few rounds, everyone is on the same page, and is able to at least attempt to win the game.

 

What goes into the process of learning how to play this game?

 

  • Language parsing: The player reading from the rule book has to turn symbols into spoken language. The players listening to the rules being read aloud have to parse the spoken language.

 

  • Pattern recognition: The players have to connect the words being read aloud with the objects in the game. “Twelve sided die” and “red soldier” have to be identified based on linguistic cues. If the instruction booklet has illustrations, these have to be matched with the real-world objects. During the game, the players have to recognize juxtapositions of pieces and cards, and key sequences of events. Good players also learn to recognize patterns in each others’ play, effectively creating models of other people’s mental states.

 

  • Motor control: The players have to be able to move pieces and cards to their correct locations on the board.

 

  • Rule-following and rule inference: The players have to understand the rules and check if they have been applied correctly. After the basic rules have been learned, good players should also be able to discover higher-level rules or tendencies that help them win. Such inferences strongly related to the ability to model other people’s’ minds. This is known in psychology as theory of mind.

 

  • Social etiquette: The players, being friends, have to be kind to each other even if some players make mistakes or disrupt the proceedings. (of course we know this doesn’t always happen).

 

  • Dealing with interruptions: If the doorbell rings and the pizza arrives, the players must be able to disengage from the game, deal with the delivery person, and then get back to the game, remembering things like whose turn it is.

 

There has been at least some progress in all of these sub-problems, but the current explosion of AI/ML is primarily a result of advances in pattern recognition. In some specific domains, artificial pattern recognition now outperforms humans. But there are all kinds of situations in which even pattern recognition fails. The ability of AI methods to recognize objects and sequences is not yet as robust as human pattern recognition.

Humans have the ability to create a variety of invariant representations. For example, visual patterns can be recognized from a variety of view angles, in the presence of occlusions, and in highly variable lighting situations. Our auditory pattern recognition skills may be even more impressive. Musical phrases can be recognized in the presence of noise as well as large shifts in tempo, pitch, timbre and rhythm.

 

AI-services-goodworklabs

 

No doubt AI will steadily improve in this domain, but we don’t know if this improvement will be accompanied by an ability to generalize previously-learned representations in novel contexts.

No currently-existing AI game-player can parse a sentence like “This game is like Settlers of Catan, but in Space”. Language-parsing may be the most difficult aspect of AI. Humans can use language to acquire new information and new skills partly because we have a vast store of background knowledge about the world. Moreover, we can apply this background knowledge in exceptionally flexible and context-dependent ways, so we have a good sense of what is relevant and what is irrelevant.

Generalization and re-use of old knowledge are aspects of a wider ability: integration of multiple skills. It may be that our current approaches do not resemble biological intelligence sufficiently for large-scale integration to happen easily.

 

 

The Essential Difference Between Couchbase & MongoDb

Coucbase Vs MongoDb

 

Couchbase and MongoDB are both document situated databases. They both have a report as their stockpiling unit. That is basically where the similarities stop. 

Couchbase is a blend of couchdb + membase. It utilizes a strict HTTP convention to question and communicate with objects. Items (reports) are put away in basins. To question records in Couchbase, you characterize a view with the segments of the report you are keen on called the guide; and after that alternatively can characterize some total capacities over the information i.e the decrease step. 

In the event that you are putting away client information and need to inquiry all clients that have not purchased any products for as far back as three months; you would first need to compose a view (the guide) that channels these clients; once this view is distributed – couchbase will improve seeks on this and you can utilize this view (outline) your source on which you execute questions. 

 

mongodb

 

You can make numerous perspectives over your reports and these perspectives are profoundly enhanced by the framework and are just reindexed when the basic record has noteworthy changes. MongoDB has a completely extraordinary way to deal with a similar issue. 

It has an idea of SQL-like inquiries, and databases and accumulations. In MongoDB, records live in an accumulation, and accumulations are a piece of a database. Much the same as Couchbase, you can store any subjectively settled report; and simply like Couchbase a programmed key is created for you.

Be that as it may, with MongoDB the way you recover archives is more similar to how you compose SQL inquiries; there are administrators for most boolean matches, and example coordinating and (with 3.0) full content hunt also. You can likewise characterize lists to help accelerate your outcomes.

 

An In Depth Analysis

 

Generally, MongoDB is less demanding to get acquainted with on the off chance that you are now alright with conventional SQL. MongoDB additionally gives the typical replication abilities and it is fit for ace replication (albeit such an arrangement isn’t empowered as a matter of course). It can most effortlessly supplant your customary social database needs; as it has similar ideas of keys/tables (“accumulations”) and question parameters – alongside the advantage of being without schematic heavy.

Couchbase and MongoDB both give business support to their databases – MongoDB’s business offering is called MongoDB Enterprise and Couchbase has Enterprise Edition (EE).

One distinction you’ll instantly discover amongst MongoDB and Couchbase is that MongoDB does not accompany a default organization reassure/GUI – in certainty a GUI and an entire facilitated administration benefit is offered as an installment choice.

 

Couchbase-Goodworklabs-comparison

 

You can introduce any number of outsider GUI to rapidly peruse your archives, yet having one as a matter of course would have been decent. Couchbase furnishes an astounding GUI with their free item.

Regarding simultaneousness, the Couch base server is both hopeful and in addition cynical locking while the MongoDB server is of both idealistic and skeptical bolting yet with a discretionary store machine called as the WiredTiger.

MongoDB’s work quality quickly embarrasses with developing number of clients. MongoDB can’t engage a ton of clients, the occasion the quantity of client builds, MongoDB begins performing contrarily. We have to include more devices for serving a great deal of clients through MongoDB which is truly exorbitant. Couchbase can bolster countless with the single hub without influencing its execution by any means.

The Couchbase server has the limit of putting away paired esteems till around 20 MB, yet the MongoDB server has a definitive limit of putting away gigantic records into an enormous number of archives. Despite the fact that the MongoDB server can store bigger pairs, yet one can keep on using Couchbase server with a different stockpiling administration for holding metadata on the doubles.

The Couchbase pieces the information and after that checks on a level plane by spreading hash space to every one of the hubs in the group of the information. The setting of the hash space to a specific hub is chosen by the key present in each report. For fracture of the information utilizing the MongoDB, a dividing strategy, and a key must be chosen since its information display is altogether archive base. This section key will reveal to you the correct area of the archive in the group.

The contrast between the two is that the MongoDB depends on us for picking the section key and the dividing technique, while the couchbase server does all the dividing independent from anyone else without human intercession.

In the end, it all depends on your requirements and the amount of flexibility your business can afford or needs. 

 

Big Data Vs Hadoop Vs Data Science

The Three Pillars Of Modern Technology

 

These three terms have been doing the tech rounds now for a long time, and most of us think that they are quite similar to each other. However, therein lies the basic difference between these emerging platforms.

Let us understand these platforms better to acknowledge the essential differences and their usage.

 

Top 10 traits of a good programmer

Apache Hadoop

Hadoop is an Apache open source framework written in java that allows distributed processing of large datasets across clusters of computers using simple programming models. A Hadoop frame-worked application works in an environment that provides distributed storage and computation across clusters of computers. Hadoop is designed to scale up from single server to thousands of machines, each offering local computation and storage.

 

Big Data

Big data means really a big data, it is a collection of large datasets that cannot be processed using traditional computing techniques. Big data is not merely a data, rather it has become a complete subject, which involves various tools, techniques and frameworks.

 

Data Science

Data science is a multidisciplinary blend of data inference, algorithm development, and technology in order to solve analytically complex problems.

At the core is data. Piles of raw information, streaming in and stored in enterprise data warehouses. Much to learn by mining it. Advanced capabilities we can build with it. Data science is ultimately about using this data in creative ways to generate business value.

 

Understanding Big Data

 

Big Data is a huge collection of data sets that can’t be store in a traditional system.

Big data is a complex sets of data. It’s size can be vary up to peta-bytes.

According to Gartner – Big data is huge-volume, fast-velocity, and different variety information assets that demand innovative platform for enhanced insights and decision making.

A Revolution, authors explain it as – Big Data is a way to solve all the unsolved problems related to data management and handling, an earlier industry was used to live with such problems. With Big data analytics, you can also unlock hidden patterns and know the 360-degree view of customers and better understand their needs.

Big data gets generated in multi terabyte quantities. It changes fast and comes in varieties of forms that are difficult to manage and process using RDBMS or other traditional technologies. Big Data solutions provide the tools, methodologies, and technologies that are used to capture, store, search & analyze the data in seconds to find relationships and insights for innovation and competitive gain that were previously unavailable.

80% of the data getting generated today is unstructured and cannot be handled by our traditional technologies. Earlier, an amount of data generated was not that high. We kept archiving the data as there was just need of historical analysis of data. But today data generation is in petabytes that it is not possible to archive the data again and again and retrieve it again when needed as Data scientists need to play with data now and then for predictive analysis unlike historical as used to be done with traditional.

 

Understanding Hadoop

 

Hadoop is an open source, Scalable, and Fault tolerant framework written in Java. It efficiently processes large volumes of data on a cluster of commodity hardware. Hadoop is not only a storage system but is a platform for large data storage as well as processing.

It provides an efficient framework for running jobs on multiple nodes of clusters. Cluster means a group of systems connected via LAN. Apache Hadoop provides parallel processing of data as it works on multiple machines simultaneously.

 

What is Data Science?

Data Science is a field that encompasses related to data cleansing, preparation, and analysis. Data science is an umbrella term in which many scientific methods apply. For example mathematics, statistics, and many other tools scientists apply to data sets. Scientist applies the tools to extract knowledge from data.

It is a tool to tackle Big Data. And then extract information from it. First Data scientist gathers data sets from multi disciplines and compiles it. After that, apply machine learning, predictive and sentiment analysis. Then sharpen it to a point where he can derive something. At last, he extracts the useful information from it.

Data scientist understands data from a business point of view.His work is to give the most accurate prediction. He takes charge of giving his predictions. The prediction of data scientist is very accurate. It prevents a businessman from future loss.

 

Although, these three tech platforms are related, but there is a major difference between them. Understanding them clearly can help us exploit and appreciate them better. 

 

Understanding IBM Bluemix

IBM Bluemix – A Mix Of Amazing Things

IBM Bluemix is a cloud platform as a Service based on open standards such as Cloudfoundry, Docker & Openstack. Bluemix comes in 3 deployment models – Public , Dedicated & Local. Bluemix Public is a multi-tenant where as Dedicated & Local are single tenant environments. Dedicated resides on IBMs Softlayer datacenter and Local resides behind your firewall. You have choice of deployment models which may not be available with Amazon or google.

Many enterprises have hybrid cloud requirements so Bluemix and other IBM Cloud offerings would be able to offer end to end solutions.

Watson based cognitive services, integrated DevOps , range of mobile backend services , database & analytics services makes IBM Bluemix a very rich platform. There are over 150+ services on the Bluemix platform.

Its very difficult to compare pricing as most of the public PaaS offerings are never easy to compare. For example, the cost of Bluemix Services are based on API calls (& storage) and there are no charges for data traffic. Many vendors may charge you for the data going out of their datacenter.

Recently IoT became has become a big tech to invest in. It is far easy to use BlueMix with Arduino or Raspberry Pi than anything else. Number of projects on Github, documentation, community help on Stackoverflow are enormous.

Advantages Of IBM Bluemix

 

1. Simplicity and speed: By focusing on the DevOps model, Bluemix can reduce the downtime of redeploying applications. Continuous delivery is one way this can be provided. The integrated environment provided by Bluemix allows developers to automatically deliver code without the hassle of building and debugging installation scripts. This reduces the time needed to manage code delivery and puts it in the hands of the testers and user community faster. The application can be deployed to multiple spaces which allow segregation of environments for development, testing and production. Automatically delivering code keeps developers focused on coding, not installation.

2. Agility: Bluemix allows developers to focus on delivering business value, rather than on maintaining the development environment, by scaling environments elastically based on business demand. Instead of manually deploying workloads, Bluemix will automatically redeploy workloads to other virtual machines (VMs) if there is an outage. To provide continuous availability, Bluemix abstracts the underlying architecture and keeps the manageability of services and applications at an easily understood level. Users are able to stop or start applications and define how much memory is associated with each application while Bluemix manages the rest.

3. Tools: With Bluemix, developers have the freedom to choose the development tools that work best for them. Developers don’t always want to work with the same tool sets and Bluemix provides several options, including the following:

  • Command line: The Cloud Foundry (CF) command line provides integration for developers that prefer coding without an integrated development environment (IDE). This is also helpful for developing automation scripts with Bluemix. The CF application programming interfaces (APIs) can be integrated with multiple languages, frameworks and services.
  • Eclipse: Since Eclipse is widely used by developers, they can continue to use the tools with which they are comfortable. The Cloud Foundry integration can be installed from the Eclipse Marketplace. This provides integration with Bluemix from the Eclipse client.
  • Web IDE: Developers can work with the Web IDE directly in Bluemix. This allows modification of the application without any development environment installed on the developers’ laptops.

4. Source control: Bluemix also comes with integration to several source control management (SCM) systems. These include Git, GitHub and Jazz SCM. These environments can be configured to deliver application changes continuously. Open source Cloud Foundry applications can be forked and loaded to Bluemix. This provides a great place to start development of a new project.

5. Services marketplace: Services leverage APIs and software development kits (SDKs) that can quickly and easily be incorporated with Bluemix applications. Although IBM provides many services, Bluemix offers an open and flexible ecosystem which allows other companies to provide services that can be integrated into applications. Companies can be both providers and users of services. Two types of services are available:

The “Managed Service Offering” is not available in the beta, but will be available soon. Services in the marketplace can be at no charge or have a pay as you go (PAYG) pricing model. These services are available to all Bluemix customers.

“User Provided Services” can be added so that organizations can share services within their organization. This promotes more reuse and standardization of services within the company.

 

The Natural Language processing (NLP) Paradigm | Big Data

The NLP Paradigm

 

The Linguistic Aspect Of Natural Language Processing (NLP)

 

Natural Language Processing is concerned with the exploration of computational techniques to learn, understand and produce human language content. NLP technologies can assist both human-human communication and human-machine communication, and can analyse and learn from the vast amount of textual data available online.

However, there are a few hindrances to this vastly unexplored aspect of technology.

We don’t consciously understand language ourselves as Homo Sapiens to begin with. The second major difficulty is ambiguity.

Computers are extremely good at manipulating syntax, for example, count how many times the word and appears in a 120 pages document, but they are extremely weak at manipulating concepts. As a matter of fact, a concept is totally stranger to computer processes. On the other hand, natural language is all about concepts and it only uses syntax as a transient means to get to it.

 

NATURAL LANGUAGE PROCESSING PARADIGM

 

A computer is unaware about conceptual processing dimension makes it difficult to process natural language since the purpose of natural languages is to convey concepts and syntax is only used as a transient means in natural language.

Such a limitation can be alleviated by making computer processes more aware about the conceptual dimension.

This is almost a philosophical question. In natural language, syntax is a means, and concept is the goal. If you relate to transportation for example, a road is the means where getting from point A to point B is the goal. If extra-terrestrial would come to earth long before we are gone and would find roads all over the place, would they be able to make some sense about transportation just by analyzing the means? Probably not! You can’t analyze the means exclusively in order to fully understand an object of knowledge.

When you think of a linguistic concept like a word or a sentence, those seem like simple, well-formed ideas. But in reality, there are many borderline cases that can be quite difficult to figure out.

For instance, is “won’t” one word, or two? (Most systems treat it as two words.) In languages like Chinese or (especially) Thai, native speakers disagree about word boundaries, and in Thai, there isn’t really even the concept of a sentence in the way that there is in English. And words and sentences are incredibly simple compared to finding meaning in text.

The thing is, many, many words are like that. “Ground” has tons of meanings as a verb, and even more as a noun. To understand what a sentence means, you have to understand the meaning of the words, and that’s no simple task.

The crazy thing is, for humans, all this stuff is effortless. When you read web page with lists, tables, run on sentences, newly made up words, nouns used as verbs, and sarcasm, you get it immediately, usually without having to work at it.

Puns and wordplay are constructs people use for fun but they’re also exactly what you’d create if you were trying your best to baffle an NLP system. The reason for that is that computers process language in a way totally unlike humans, so once you go away from whatever text they were trained on, they are likely to be hopelessly confused. Whereas humans happily learn the new rules of communicating on Twitter without having to think about it.

If we really understood how people understand language, we could maybe make a computer system do something similar. But because it’s so deeply buried and unconscious, we resort to approximations and statistical techniques, which are at the mercy of their training data and may never be as flexible as a human.

Natural language processing is the art of solving engineering problems that need to analyze or generate natural language text.The metric of success is not whether you designed a better scientific theory or proved that languages X and Y were historically related. Rather, the metric is whether you got good solutions on the engineering problem.

For example, you don’t judge Google Translate on whether it captures what translation “truly is” or explains how human translators do their job. You judge it on whether it produces reasonably accurate and fluent translations for people who need to translate certain things in practice. The machine translation community has ways of measuring this, and they focus strongly on improving those scores.

When is NLP used?

NLP is mainly used to help people navigate and digest large quantities of information that already exist in text form. It is also used to produce better user interfaces so that humans can better communicate with computers and with other humans.

Saying that NLP is engineering, we don’t mean that it is always focused on developing commercial applications. NLP may be used for scientific ends within other academic disciplines such as political science (blog posts), economics (financial news and reports), medicine (doctor’s notes), digital humanities (literary works, historical sources), etc.

Although, it is being used also as a tool within computational X-ology in order to answer the scientific questions of X-ologists, rather than the scientific questions of linguists.

That said, NLP professionals often get away with relatively superficial linguistics. They look at the errors made by their current system, and learn only as much linguistics as they need to understand and fix the most prominent types of errors. After all, their goal is not a full theory but rather the simplest, most efficient approach that will get the job done.

NLP is a growing field and despite many hindrances, it has come forward and shown us tremendous capabilities to abstract and utilize data. It teaches us that simplicity is the key at the end of the day. 

 

Understanding Amazon Web Services

The Revolution Called AWS

 

Let’s switch back to the old ages.

Imagine you are heading overseas for your travel endeavors.

You have options to go anywhere but wherever you go, you need a place to stay.

 

Traditionally, you would have to build your own shelter upon arrival. You would cut wood for the walls, collect leaves to build shade, and segregate compartment for rooms as required. You would then build a kitchen, toilets, and bed as additional features in the house. At the finest day, your activities will be cooking, sleeping, drinking and showering.

Before you realize it, your holiday has finished and it’s time to come back to reality.

Soon, many great builders arise because of the huge demand for shelters. When you go for a holiday, you could pay to have someone build a house for you. However this is very expensive unless you decide to live at that new place for a long time. And keep in mind disasters do happen. Your floor sinks and your builder runs away with your money, the house is not what you ordered for and now you are in a complete dilemma in an unknown land.

Soon, a service known as ‘hotel’ is created. It provides shelter at locations of your liking, different pricing models based on number of guests, nights, and your preferred exclusivity. In addition, housekeeping service are there to ensure your tenacity in the hotel is at its best. So you always get clean bed sheets, bathroom with toiletries, and alcohol in the fridge for your disposal. Now here comes the hardest choice, pick one.

The great thing about this is, you do not need to know how to build a wall, start a fire, do cooking and so on. The amount you pay is proportional to your chosen preferences and exclusivity. Best of all, you focus on enjoying your holiday.

With this model, it is very obvious that your risks are mitigated, you spend only what is required, and are able to focus on your holiday. Honestly, hotel services are a great innovation!

AWS stack is the hotel service provider in the virtual world, only more powerful. They have expertise in developing and managing a large scale “physical” infrastructures, eg: EC2 (walls, shade, pipes), skillful in providing “features” platform services like VPC, Lambda, RDS, Auto-Scaling and Load Balancing (kitchen, toilet, bedroom, garden). You focus on your main business, they focus on theirs; there is no huge upfront cost; people can build fast, fail fast with minimal cost, and to top it all, it is an extremely powerful component for innovation to happen.

 

“Amazon Web Services (AWS) is a secure cloud services platform, offering compute power, database storage, content delivery and other functionality to help businesses scale and grow.”

 

Services Offered By AWS

 

Let us have a look at the various services offered by AWS and their brief descriptions.

 

Amazon Elastic Compute Cloud (EC2)

Allows renting a server per hour (from small to huge.)

Amazon Elastic MapReduce

Allows renting a couple of servers per hour as an hadoop cluster (to crunch a lot of data.)

Auto Scaling

Allows renting just enough servers as necessary (using more with a lot of work, using less with less work.)

Amazon CloudFront

Facilitates something like Akamai services (CDN, content distribution network) but at much cheaper rates.

Amazon SimpleDB

Allows noSQL for key-value store, store huge lists of items consisting of attributes that are quickly searchable.

 

Amazon Relational Database Service (RDS)

MySQL as a Service is one of the best services of AWS at this moment which kills redundant data and provides complete data superiority. 

 

AWS Elastic Beanstalk

Allows infrastructure automation for java applications. works with tomcat, uses services like autoscaling. (turn-key solution, perfect for getting familiar with several AWS services.)

 

AWS CloudFormation

Allows infrastructure orchestration. write recipes for different services you want to use, and how, and CloudFormation will do the rest.

 

Amazon Fulfillment Web Service (FWS)

Have a shop? You can have Amazon manage your fulfillment. your suppliers ship to Amazon, and they ship to your customers.

 

Amazon Simple Queue Service (SQS)

The one of a kind ‘first in, first out’ (in nature) messaging, perfect for jobs that should be processed asynchronously, more or less in order of arrival. think of video rendering or thumbnail generation.

 

Amazon Simple Notification Service (SNS)

Publish/subscribe mechanism, you create topics interested parties can subscribe to, sending a message to a topic, sends the message to all subscribers (like a mailing list, but for software systems.)

 

Amazon Simple Email Service (SES)

A service for sending mail. You have to earn your quota with a good behaviour. An interesting way of minimizing spam.

 

Amazon CloudWatch

Alows monitoring of many AWS assets, like instances (servers) and load balancers.

 

Amazon Route 53

DNS  facilities with amazing flexibility.

 

Amazon Virtual Private Cloud (VPC)

Allows extension of your private network with ‘some cloud’.

 

Elastic Load Balancing

Allows distributing traffic over one or more instances (servers).

 

Amazon Simple Storage Service (S3)

The most revolutionary service in AWS. unlimited storage.

 

Amazon Elastic Block Store (EBS)

The network disks for AWS, attaches a volume to an instance of your choice. You can’t share the disk between multiple instances.

 

AWS Import/Export

Allows moving large data in and out of S3.

 

Alexa Top Sites

Display top sites from Alexa.com in your application.

 

Identity and Access Management (IAM)

Lets you provision users within your AWS account and grant them access to different services and resources.

 

At GoodWorkLabs, we understand Amazon Web Services in & out. We understand the intricacies involved and the minute details that make the implementation process hassle free.

Contact us today for world class AWS services.

 

Ready to start building your next technology project?