Data Warehousing in the Cloud
Why is cloud so important?
Data warehouses are currently going through two very significant transformations that have the potential to drive significant levels of business innovation.
The first area of transformation is the drive to increase overall agility. The vast majority of IT teams are experiencing a rapid increase demand for data. Business teams want access to more and more historical data whilst at the same time, data scientists and business analysts are exploring ways to introduce new data streams into the warehouse to enrich existing analysis as well as drive new areas of analysis. This rapid expansion in data volumes and sources means that IT teams need to invest more time and effort ensuring that query performance remains consistent and they need to provision more and more environments (data sandboxes) for individual teams so that they can validate the business value of new data sets.
The second area of transformation is around the need to improve the control of costs. There is a growing need to do more with fewer and fewer resources whilst ensuring that all sensitive and strategic data is fully secured, throughout the whole lifecycle, in the most cost efficient manner. Cloud is proving to be the key enabler. It allows organisations to actively meet the challenges presented by the two key transformations of expanding data volumes and increased focus on cost control.
In this series of blog posts I hope to explain why and how moving your data warehouse to the cloud can support and drive these two key transformation as well as explaining the benefits that it brings for DBAs, data scientists and business users. Hopefully, this information will be useful for enterprise architects, project managers, consultants and DBAs. Over the coming weeks I will cover the following topics:
1. Why is cloud so important for data warehousing
2. Top 3 use cases for moving your data warehouse to the cloud
3. Oracle’s cloud solutions for data warehousing
4. Why Oracle Cloud?
5. Why Oracle’s Cloud runs Oracle better
6. A review of Oracle’s complete architecture for supporting data warehousing in the cloud
In this first post I will cover points 1 and 2 and subsequent posts then cover the other topics. So here we go with part 1…
Why move to the cloud now?
A recent report by KPMG (Insights & Predictions On Disruptive Tech From KPMG’s 2015 Global Innovation Survey: https://softwarestrategiesblog.com/tag/idc-saas-forecasts/) looked at all the various technologies that are likely to have the greatest impact on business transformation over the coming years. KPMG talked to over 800 C-level business leaders around the world from a very broad range of businesses including tech industry startups, mid to large-scale organizations, angel investors and venture capital firms.
One of the key objectives of the survey was to identify disruptive technologies and innovation opportunities. Looking forward, the top 3 technologies that will have the biggest impact of business transformation are: cloud, data and analytics and Internet of Things. All three of these technologies are key parts of the today’s data warehouse ecosystem. Therefore, it is possible to draw the conclusion that technology leaders view data warehousing in the cloud as having the greatest potential for driving significant business impact.
The importance of cloud for data warehousing to Oracle customers is directly linked to three key drivers:
• Increased agility
• Better cost control
• Co-location
Improving agility
Many data warehouses are now embarking on a refresh phase. With much of the ground work for working with big data now in place, businesses are looking to leverage new data streams and new, richer types of analytics to support and drive new project areas such as: Customer-360, predictive analytics, fraud detection, IoT analytics and establishing data as profit center. Many of these projects require provisioning of new hardware environments and deployment of software. It is faster, easier and more efficient to kick-start these new data centric projects using Oracle’s comprehensive Cloud Services.
Delivering better cost control
Many IT teams are looking for ways to consolidate existing Oracle marts, each running on dedicated hardware, and legacy non-Oracle marts, running on proprietary hardware, into a single integrated environment. The delivery of Oracle’s enterprise-grade cloud services provides the perfect opportunity to start these types of projects and Oracle has cloud-enabled migration tools to support these projects. Compliance cannot be seen as an optional extra when planning a move to the cloud. Data assets need to be secured across their whole lifecycle. Oracle’s enterprise-grade cloud services make compliance easier to manage and more cost efficient because all security features can be enabled by default and transparently upgraded and enhanced.
Co-Location for faster loading
The vast majority of Oracle E-Business customers have already begun the process of moving their applications to Oracle’s Cloud Services. Most data warehouses source data directly from these key applications such as order entry, sales, finance and manufacturing etc. Therefore, it makes perfect sense to co-locate the data warehouse alongside source systems that are already running in the cloud. Co-location offers faster data loading which means that business users get more timely access to their data.
Key benefits of moving to the Oracle Cloud
There are typically three main benefits for moving the data warehouse to Oracle Cloud and these are directly linked to the three key drivers listed in the previous section:
1. Easier CONSOLIDATION and RATIONALIZATION
2. Faster MONETIZATION of data in the Cloud
3. Cloud offers better PROTECTION
Let’s explore each of these use cases in turn:
1) Easier consolidation and rationalization
“All enterprise data will be stored virtually in the cloud. More data is in the cloud now than in traditional storage systems” – Oracle CEO Mark Hurd
For a whole variety of reasons many data warehouse environments are made up of a number of databases that cover corporate data sets, departmental data sets, data discovery sandboxes, spread-marts etc. Each one of these usually runs on dedicated hardware that requires on-going investment and dedicated DBA services. This means that a significant proportion of IT costs is allocated to just keeping the lights on rather then helping drive new levels of innovation.
Oracle customers are seeing the opportunities provided by the Oracle Multitenant feature of Database 12c and the availability of Oracle’s Cloud Services for data warehousing as an ideal opportunity to consolidate systems into a single cloud environment. At the same time these systems benefit from co-location: being next to their key source systems, which are already running in the cloud. By moving to the Oracle Cloud it is possible to both improve ETL performance and reduce IT costs.
With growing interest in big data there is an on-going need to rapidly provision new sandboxes for data discovery projects. The provisioning process is much simpler and faster using Oracle’s Cloud Services, allowing business teams to start their discovery work a lot quicker.
As new data warehouse projects are formulated the deployment of development, test and training environments within the cloud provides the ability to free up costly on-premise resources and make them available to other projects.
The cloud provides and excellent opportunity to convert and transform outdated legacy marts by leveraging the sophisticated analytical features of Oracle’s industry leading database. Oracle has a complete set of cloud-ready migration tools to convert and move schemas and data from all major database vendors to the Oracle Cloud.
2) Faster Monetization
IoT – next wave of data explosion. By 2020 there will be 32 billion connected devices, generating 4.4ZB (Cars, Infrastructure, Appliances, Wearable Technology)
There is a growing need to monetize data, i.e. treat it as a valuable asset and convert it into a profit center. Moving to the cloud opens up a wide range of opportunities by providing an agile and simple way of implementing new style hybrid transactional/analytical requirements. It is a quick, efficient way to onboard new data streams such as IoT, external data sets from social media sources, 3rd party data sources etc. to enrich existing data sets making it possible to explore new business opportunities.
The fast deployment and tear-down capabilities of the cloud offers an effective way to keep both deployment and development costs down to support the new style of “fail-fast” data discovery projects.
The success of these data monetization projects largely depends on being able to integrate both existing traditional data sources, such as those from transactional operations, and the newer big data/IoT data streams. This integration is not only about matching specific data keys but also the ability to use a single, common industry standards driven query language such as SQL over all the data. Oracle’s Data Warehouse Cloud Services uniquely offers tight integration between relational and big data via features such as Big Data SQL.
3) Better Protection
“Oracle’s enterprise cloud will be the most secure IT environment. We are fully patched, fully secured, fully encrypted—that’s our cloud. . .“ – Oracle CEO Mark Hurd
Treating data as a profit center naturally requires IT teams to consider the concept of data protection and data availability. Outside of the cloud, security profiles have to be replicated, operating system patch levels need to kept in sync along with database patchsets. Trying to enforce common security and compliance rules across multiple standalone systems that share data is a time-consuming and costly process.
The processes of consolidating data sets by leveraging the multitenant features of Oracle Database 12c and moving schemas to Oracle Cloud Services gives DBAs a simple and efficient way to secure data across the whole lifecycle. Within Oracle’s Cloud Services all data is encrypted by default: in transit, in the Cloud and at rest. Backups with encryption are enforced which ensures that cloud data remains secure at all times.
Overall security is managed using Oracle Key Vault, secure SSH access, federated identity and an isolated cloud management network. Compliance and reporting is managed through the use of comprehensive audit trails.
Below the database level Oracle Cloud Service is fully managed by Oracle, which means it is always fully patched and therefore fully secured. This allows DBAs and business teams to focus on business innovation without having to invest considerable time and resources securing, encrypting and patching environments to ensure they are in line with compliance and regulatory requirements.
Summary
In this blog post I have examined why you need to start thinking about and planning your move to the cloud: looking forward data warehousing in the cloud is seen as having the greatest potential for driving significant business impact through increased agility, better cost control and faster data integration via co-location. I have outlined the top 3 key benefits of moving your data warehouse to the Oracle cloud: it provides an opportunity to consolidate and rationalise your data warehouse environment, it opens up new opportunities to monetise the content within your warehouse, new data security requirements means require IT teams to start implementing robust data security systems alongside comprehensive audit reporting.
In the next post I will discuss Oracle’s cloud solutions for data warehousing, how Oracle’s key technologies enable Data Warehousing in the cloud and why Oracle’s Cloud runs Oracle better than any other cloud environment.