Object Storage - NetApp Storage Grid

Præsentationer af emnet: "Object Storage - NetApp Storage Grid"— Præsentationens transcript:

1 Object Storage - NetApp Storage Grid
edgemo summit 2017 Mig – 5 år med NetApp storage som System Engineer, startede i kanalen hos distibrutøre. Hvad er Object storage Hvad er vores produkt – NetApp Storage Grid Hvad kræver og koster det at komme igang © 2017 NetApp, Inc. All rights reserved. © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

2 The evolving data management challenge
7/19/2018 The evolving data management challenge Internet of Things is driving decentralized data creation, consumption and analytics New Apps and Rich Content require different approach to scale performance and capacity Balance security, cost, and performance in the hybrid cloud GDPR compliance Den største udbyder, eller anvender af Object storage var Amazom. I dag vokseron premise Object storage faktisk hurtigere end Amazon. Volvo historie – to biler mere data end hele firmaets leve tid Primært pga nye trends, så som IoT. Udfodringer med hvor data skal leve, pga App’s som generer data som skal være tæt på brugeren. Fx mobile enheder. Hvordan styrere man værdien af dataen for brugeren eller applikationen? The largest deployment for object storage is the public cloud, where Amazon S3 (simple storage services) is the most popular example and its object storage protocol also called S3 is now an industry standard. So you will hear often developers and IT teams talking about the S3 API (application program interface). An interesting observation is that object storage on premises is growing at a faster rate than in the public cloud as IT teams look to copy the best practices developed by Amazon to manage rich content at scale and new trends like IoT. The Internet of Things not only drives massive data growth in unstructured data – it also changes how data is stored and accessed. As data is created and consumed across many sites (as opposed to a more traditional data center setup), IT departments need to reevaluate how they can manage large amounts of data spread over many different locations. This leads to multisite data stores that bring data closer to the workloads, applications, and users. The growth in unstructured data also drives new requirements for storing and protecting data. With object-enabled data management, organizations can now establish highly granular, flexible data management policies that determine how data is stored and protected, taking into consideration a wide range of performance, durability, availability, geo-location, and longevity requirements. The value of data typically changes over time, as does the cost of storing data in a particular location or storage technology. Policy-based data management enables organizations to make sure that data is stored optimally throughout the entire data lifecycle – including storing data on-premises and in off-premises - or cloud - infrastructures. © 2016 NetApp, Inc. All rights reserved. NetApp, Inc. All rights reserved. NetApp Proprietary

3 Block storage Traditionel storage er fx block storage. En spand med 0 og 1, uden storage har en ide om hvad de repræsenterer. Ingen rigtig data intelligens i storage laget, ud over snapshots og andre små integrationer. © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

4 Comparison to Block Storage
Objecterne tager form og kan nu identificeres og går nu fra at være et storage system til at være et software system. Hvilket giver os en del fordele, som jeg kommer ind på senere. Funktioner mod dataen (softwaren) da vi har kender skab til hvad objectet/dataen indeholder. © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

5 What is an Object ? Hvordan gør vi så det? – Altså hvad består et object af? Filen, eller filer, system data og så vores object data. Eller selve beskrivelsen af data, som gør det hele søgbart og i sidste endnu til brugbar data. © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

6 Metadata – data on data –think of Metadata like the label on a can
PostDanmark – labels er vigtigere end indhold. Samme er det med data. Søgbarhed og planlægning af datafremtid og lokation Spotify anvender S3 i AWS pga muligheden for at lave en stor spand. © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

7 Metadata – data on data Rettigheder Specifikationer
Log filer over redigering Hvad er det Beskrivelse Tænkt hvordan denne information kan buges? - © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

8 Metadata – data on data Det er selvfølgelig vigtigt at data/objecter selv kan tilføje denne information. Så det bliver så automatiseret som muligt. API / app skal være skrevet til at tilføje brugbar og ikke mindst søgbar metadata © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

9 Metadata – data on data – analyze data based on Metadata
Analyse af data Returnering af objecter fra softwaren ud fra søge kriterier. Det er meget nemmere at planlægge ud fra object info. Fx Anvende eksterne analyse services i skyen ved at give dem adgang til objecter med specifikke meta data © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

10 Metadata – data on data – Big Data, Compliance and Governance
Object storage er i vores øjne første step mod BigData. Samle data Udvinde information omkring data Gør det søgbart Finde trends, altså få mere indsigt i ens data Shit in shit out – data skal starte et sted © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

11 En vitgig ting er at alle object er styret ud fra politkker.
Hvor skal data bo? Hvor længe? Skal det flyttes © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use © 2015 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only

12 Use Cases for Object Storage
7/19/2018 Use Cases for Object Storage Archives Cheap & deep, low transaction loads Long retention periods, latency tolerant Criteria: high efficiency, durable, automated tiering as value of data changes Media repositories Large objects, globally distributed Streaming data access, large throughput rates Criteria: multi-site, multi-app, scalability, automated distribution Web data repositories Sensor data, images, extremely high transaction loads Billions of objects Criteria: scalability, performance, native web interfaces © 2016 NetApp, Inc. All rights reserved. NetApp, Inc. All rights reserved. NetApp Proprietary

13 What is StorageGRID Webscale?
Multiple: Applications + Sites + Protocols Hyperscale global object store Software-defined nodes Heterogeneous infrastructure Native file and object protocols Multi-tenant with AD/LDAP integration Dynamic data lifecycle management Site 1 Site 2 Site 3 NAS Applications Applications Web Applications SMBv2.x NFSv3 Swift S3 Multiple: Tenants + Policies+ Administrators StorageGRID Webscale StorageGRID Webscale is a hyperscale global object store. A single instance of StorageGRID can scale out to across 16 sites, 70 petabytes and 100 billion objects. <click> For more than a decade, StorageGRID has delivered a software-defined platform for content repositories for mission-critical applications. <click> You can deploy StorageGRID Webscale nodes on engineered hardware appliances, which are built on one of the industry’s most reliable platforms and that is the NetApp E-Series. Or you can deploy software-only nodes using virtual machines, running in VMware, or KVM on OpenStack. With virtual nodes, you can deploy NetApp or third party disk arrays underneath. StorageGRID can also use tape and S3-compatible public cloud as an integrated tier. <click> StorageGRID supports natively file and object protocols, which are SMB 2.x, NFSv3, S3 and Swift. <click> StorageGRID supports multiple tenants and provides user identity federation using AD and LDAP. <click> StorageGRID provides dynamic data lifecycle management policy. Admins can create rules that manage data lifecycle at the object level, and customize data locality, durability, performance, efficiency and retention time. Appliance NetApp Third Party Tape Public Cloud Multiple: Targets + Tiers + Sites © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use

14 Software-defined object storage
7/19/2018 Deployment options One size does not fit every datacenter Software-defined object storage Virtual machines, containers, appliances can be mixed across all sites Support for third-party block storage arrays Perpetual capacity based license SG5600 appliance 2U & 4U appliance with 12 or 60 drives Storage and compute in one chassis Available 48TB to 480TB configurations 10GbE connectivity Reduce dedicated compute requirements Optimized rack space, power, cooling Another key element of StorageGRID Webscale is data management. As part of the data management framework, StorageGRID Webscale supports multiple deployment models from virtualized software deployments in on-premise or off-premises private clouds to the support for third party storage systems along with NetApp E-Series storage systems. StorageGRID Webscale supports both S3 and OpenStack Swift protocols for seamless integration of cloud applications. The dynamic policy engine in StorageGRID Webscale executes policies against data at ingest as well as at rest, taking into considerations factors like resource availability, latency, data retention and geo location requirements as well as network cost. © 2016 NetApp, Inc. All rights reserved. NetApp, Inc. All rights reserved. NetApp Proprietary

15 Men hvad koster det? © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use

16 © 2015 NetApp, Inc. All rights reserved
© 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use