Object storage

Object storage is a storage architecture that manages data as objects, where an object is defined as a file with its metadata, and a globally unique identifier called the object ID.

Examples of metadata are filename, date and time stamps, owner, access permissions, the level of data protection, and replication settings to for instance a different geography.

Object storage stores and retrieves data using a REST API over HTTP, served by a webserver, and is designed to be highly scalable.

Where a traditional file system provides a structure that simplifies locating files (for example, a log file is stored in /var/log/proxy/proxy.log), in object storage, a file’s object ID must be administered by the application using it. Using the object ID, the object can be found without knowing the physical location of the data. For example, an application has administered that its log file is stored in object ID 8932189023.

Using object IDs enables simplicity and massive scalability of the storage system, as the object ID is a link to an object that can be stored anywhere.

Data in object storage can’t be modified. Instead, if a file is modified, the original file must be deleted, and a new file must be created, leading to a new object ID. This makes object storage unsuitable for frequenty changing data. But it is a good fit for data that doesn't change much, like backups, archives, video and audio files, and virtual machine images.
Object storage allows for high availability using commodity servers with direct attached disk drives. It can be setup to replicate objects across multiple servers and locations (typically, at least three copies of every file are stored in mutiple geographical zones). If one or more servers or disks fail, data can still be made available, without impact to the application or the end user.

While object storage was not designed to be used as a file system, some systems emulate a file system using object storage. For instance, Amazon’s S3FS creates a virtual filesystem, based on S3 object storage, that can be mounted to an operating system in the traditional way, however, with significant performance degradation. A much better solution is to use object storage with applications designed for it.

This entry was posted on Vrijdag 07 Oktober 2016