How to securely query AWS S3 from ClickHouse
ClickHouse is a powerful analytical database, loading data from an AWS S3 bucket allows you to quickly start analyzing the data without having to download and upload them manually in the database.
Writing data to S3 allows you to offload some cold data, while keeping all the hot data in ClickHouse and, with the usage of federated queries, being able to query both hot and cold data at the same time.
In this video we’ll showcase how to:
- Setup a secure connection between AWS S3 and ClickHouse
- How to read data from AWS S3 in ClickHouse
- How to write data from ClickHouse in AWS S3
Check out these resources to learn more:
- ClickHouse S3 integration: https://clickhouse.com/docs/en/integrations/s3
- Querying external data in Aiven for ClickHouse: https://docs.aiven.io/docs/products/clickhouse/concepts/federated-queries
- Read and write data in AWS S3 from Aiven for ClickHouse: https://docs.aiven.io/docs/products/clickhouse/howto/run-federated-queries
Chapters:
00:00 Intro
00:18 The dataset and the AWS settings
02:08 Get started with Aiven for ClickHouse
02:36 Connect to ClickHouse
03:31 Read data from an AWS S3 bucket
04:16 Write data to an AWS S3 bucket
05:25 Summary
#clickhouse #aws #s3 #awss3 #amazons3 #query
ABOUT AIVEN
Aiven’s cloud data platform helps your business reach its highest potential by making your data work for you.
It provides fully managed open source data infrastructure on all major clouds, helping developers focus on what they do best: innovate and create without worrying about the limitations of technology.
We like to think that Aiven is not only a cloud data platform but also an extension of your team. We are dedicated to helping you to succeed by removing barriers and finding the right solutions – with the help of the best data technology there is.
Try Aiven for free: https://console.aiven.io/signup
CONNECT WITH US
Website: http://aiven.io
LinkedIn: https://linkedin.com/company/aiven
GitHub: https://github.com/aiven
Twitter: https://twitter.com/aiven_io