Showing posts with label avro. Show all posts
Showing posts with label avro. Show all posts

Thursday, 29 June 2017

What is Avro Seriealization

*Avro-Serialization is the process of translating data structures or objects state into binary or textual form to transport the data over network or to store on some persisten storage.

*Once the data is transported over network or retrieved from the persistent storage, it needs to be deserialized again. Serialization is termed as marshalling and deserialization is termed as unmarshalling.

Features of Avro

Features of Avro

* Avro is a language-neutral data serialization system.

* It can be processed by many languages (currently C, C++, C#, Java, Python, and Ruby).

* Avro creates binary structured format that is both compressible and splittable. Hence it can be efficiently used as the input to Hadoop MapReduce jobs.

* Avro provides rich data structures. For example, you can create a record that contains an array, an enumerated type, and a sub record. These datatypes can be created in any language, can be processed in Hadoop, and the results can be fed to a third language.

* Avro schemas defined in JSON, facilitate implementation in the languages that already have JSON libraries.

* Avro creates a self-describing file named Avro Data File, in which it stores data along with its schema in the metadata section.

* Avro is also used in Remote Procedure Calls (RPCs). During RPC, client and server exchange schemas in the connection handshake.

What is avro

* Apache Avro is a language-neutral data serialization system. It was developed by Doug Cutting, the father of Hadoop. Since Hadoop writable classes lack language portability, Avro becomes quite helpful, as it deals with data formats that can be processed by multiple languages. Avro is a preferred tool to serialize data in Hadoop.

* Avro has a schema-based system. A language-independent schema is associated with its read and write operations. Avro serializes the data which has a built-in schema. Avro serializes the data into a compact binary format, which can be deserialized by any application.

* Avro uses JSON format to declare the data structures. Presently, it supports languages such as Java, C, C++, C#, Python, and Ruby.