HBase Schema Design - Things you need to know

O'Reilly · 129K views · 2011 · 1:12:36

Open on YouTube

Databases & storageDistributed systemstalk

Why Stack Overflow recommends it

Stack Overflow users link it when answering or asking about HBase row key design, region splitting and schema layout, since it explains the storage internals that drive those choices.

Recommended by 3 people across 3 different months, 2015 to 2017.

What it is

A 2011 conference session by Lars George, an HBase committer then at Cloudera, on how to design schemas for HBase. It covers the storage layer concepts behind row keys, regions and column families, and how schema choices affect performance when building new applications or porting from relational databases.

What people said

Recommends Les Vogel · 2015-08 · Stack Overflow · on “What's the general schema that Gmail uses with Bigtable”

Schema design is very specific to your application. HBase fortunately has quite a bit of public material to help: There is a Quora Thread [LINK] on production schemas. The Time Series [LINK] material that comes after the doc you mentioned is good. The HBase documetnation [LINK] has a chapter on Schema design. Lars George [LINK] 's HBase book [LINK] has some good discussion. He also talks on [this video] Ian Varley's HBaseCon talk [LINK] covers this as well. Amandeep Khurana has an article [LINK] .

Recommends siddhartha jain · 2016-08 · Stack Overflow · on “Does hbase really scales linearly?”

First if you want you can install HBase over any supported file system. It is not mandatory to use it over Hdfs but using it with Hdfs give advantage to it like Fault taulrence , Data replication, checksums etc. That's why it is recommended to use HBase over hdfs Moreover although there is a bottleneck of namenode in hdfs but it does not effect HBase efficiency because it is not that every operation internal working is dependent on namenode of hdfs for instance Region servers serve data for reads and writes. When accessing data, clients communicate with HBase RegionServers directly while Region assignment, DDL (create, delete tables) operations are handled by the HBase Master process. Which

Recommends Alex Woolford · 2017-06 · Stack Overflow · on “HBase schema design to store video frames: rows vs columns, rows as columns”

A friend asked me how to store raw video, frame by frame, in HBase. The typical access pattern would be to retrieve frames for a block of time. Each frame is approx. 7MB and the footage is captured at about 30 frames-per-second. A 20-minute video, for example, would take about 250GB of storage. I saw an excellent video by Lars George, author of HBase: the definitive guide [LINK] , titled HBase Schema Design: things you need to know [this video] , where he talks about storing video "chunks" (the snippet where he's talking about video starts at 1:07:12 and ends at 1:08:52), so it's seems like HBase could, potentially, be a fit for this use-case. I created a couple of row-key options: Scenario 0: ro

Cites Mark Rajcok · 2015-01 · Stack Overflow · on “reverse domain name row key, automatic splitting, and load balancing”

I'm designing an HBase schema with a row key that starts with the domain name reversed. E.g., com.example.www . Although there are many more domains that end in .com than say .org or .edu , I assume that I don't have to manage splitting myself, and I can rely on HBase's automatic splitting to distribute the rows across regions. I.e., regions will split as they get too large. I should end up with more regions that have keys that start with com. than say org. , but I assume that's okay, and the " com. regions" should end up distributed across my region servers, correct? Is there an issue with load balancing here? In the 2011 HBase Schema Design video [this video] by Lars (the link goes directly to