One of the most important aspect of architecting a solution with Big Data is choosing a proper Data Storage options in Hadoop/Spark. Hadoop does not have a standard data storage format ,but as a standard file system ,allows for storage of data in any format ,whether it’s text,binary ,image or other.
When working with Hadoop Framework to process massive data, we will face many challanges and difficulties which includes the Input/Output(I/O) and network related bottlenecks. To overcome this problem, one needs to understand the different data compression techniques available in Hadoop Framework.
Function is a block of executable code that returns a value . Function in scala can be defined inside another function .It can be returned as an output from another function.
When working with Apache spark ,there are times when you need to trigger a Spark job on demand from withing and outside the cluster.There are two ways in which we can submit Apache spark job in a cluster which includes bash script and REST API.
Once you start working on Linux and writing a Shell script, you will notice that all files and directories on Linux have a standard set of access permissions. These access…
SQL or Structured Query Language is a special-purpose programming language designed for managing data held in a relational database management system (RDBMS). It provides various ways of manipulating and creating…
Machine Learning is a field of computer science that uses statistical techniques to train a computer system to learn from data and act according to it.
In this blog post we will install Apache Hive in Ubuntu Machine.Once installation is complete we will run Hive queries using Hive Query Language(HQL) to Verify the installation.
A single cancer patient on average can generate 1 Terabyte of biomedical diagnostic and clinical data. According to a paper from GLOBCAN 2018, there will be 18.1 million new cancer…
Blockchain provides a trusted and distributed ledger with shared business process and parties.It allows any participant in the business network to see the one system of record or ledger.