Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Documentation

on:
push:
branches: [master]
pull_request:
branches: [master]

permissions:
contents: read

jobs:
docs:
runs-on: ubuntu-latest
permissions:
contents: write
pages: write
services:
clickhouse:
image: clickhouse/clickhouse-server:26.9
env:
CLICKHOUSE_SKIP_USER_SETUP: '1'
ports:
- 9000:9000
options: >-
--health-cmd="clickhouse-client --query 'SELECT 1'"
--health-interval=5s
--health-timeout=5s
--health-retries=12
steps:
- uses: actions/checkout@v6
- uses: julia-actions/setup-julia@v2
with:
version: '1'
- uses: julia-actions/cache@v2
- name: Install documentation dependencies
shell: julia --color=yes --project=docs {0}
run: |
using Pkg
Pkg.develop(PackageSpec(path=pwd()))
Pkg.instantiate()
- name: Build and deploy documentation
run: julia --color=yes --project=docs docs/make.jl
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Publish GitHub Pages
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
run: gh api --method POST "repos/${{ github.repository }}/pages/builds"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 4 additions & 0 deletions docs/Project.toml
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
[deps]
ClickHouse = "82f2e89e-b495-11e9-1d9d-fb40d7cf2130"
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"

[compat]
Documenter = "1.19"
14 changes: 14 additions & 0 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,23 @@ using ClickHouse
makedocs(
modules = [ClickHouse],
sitename = "ClickHouse.jl",
checkdocs = :exports,
pagesonly = true,
format = Documenter.HTML(
prettyurls = true,
canonical = "https://juliadatabases.org/ClickHouse.jl/",
edit_link = "master",
footer = "Powered by [Documenter.jl](https://documenter.juliadocs.org/). AI disclosure: This work was prepared with assistance from OpenAI Codex.",
),
pages = [
"index.md",
"usage.md",
"api.md",
]
)

deploydocs(
repo = "github.com/JuliaDatabases/ClickHouse.jl.git",
devbranch = "master",
versions = nothing,
)
1 change: 1 addition & 0 deletions docs/src/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ ClickHouseSock

```@docs
connect
connect!
execute
insert
ping
Expand Down
10 changes: 10 additions & 0 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,22 @@ Usage examples can be found on the [usage page](@ref Usage).

## Limitations

- Connections use the unencrypted native TCP protocol, normally on port 9000.
HTTPS and encrypted native TCP endpoints require a different client.
- Timezone conversion of `DateTime` / `DateTime64` for columns that have a
timezone assigned in ClickHouse doesn't happen automatically. All DateTime
objects are naive, meaning they aren't timezone aware. For reasoning, see
[this post](https://github.com/JuliaDatabases/ClickHouse.jl/pull/21) and
[this post](https://github.com/JuliaDatabases/ClickHouse.jl/issues/7#issuecomment-683311706).

## Compression

The `compression` keyword of [`connect`](@ref) selects the outgoing block format:
`ClickHouse.COMPRESSION_NONE`, `ClickHouse.COMPRESSION_LZ4`,
`ClickHouse.COMPRESSION_CHECKSUM_ONLY`, or `ClickHouse.COMPRESSION_ZSTD`.
The client reads compressed replies using the format recorded in each block;
the server chooses that reply format.

## Index

```@index
Expand Down
87 changes: 38 additions & 49 deletions docs/src/usage.md
Original file line number Diff line number Diff line change
@@ -1,90 +1,79 @@
# Usage

```@meta
DocTestSetup = quote
using ClickHouse
end
## Connecting

Connect to the server's native TCP port, normally 9000. This client uses
unencrypted native TCP. It cannot connect to an HTTPS endpoint or an encrypted
native TCP endpoint, such as the [ClickHouse playground](https://clickhouse.com/docs/get-started/sample-datasets/playground).

```@repl usage
using ClickHouse
sock = ClickHouse.connect("localhost", 9000; compression = ClickHouse.COMPRESSION_LZ4);
```

## Executing DDL

### Creating a table
```jldoctest
execute(connect(), """
CREATE TABLE IF NOT EXISTS MyTable

This temporary table lasts until the connection closes. Reuse the same socket
for the following queries.

```@repl usage
ClickHouse.execute(sock, """
CREATE TEMPORARY TABLE MyTable
(u UInt64, f Float32, s String)
ENGINE = Memory
""")

# output

```

## Inserting data
```jldoctest
insert(connect(), "MyTable", [Dict(

```@repl usage
data = Dict(
:u => UInt64[42, 1337, 123],
:f => Float32[0., ℯ, π],
:s => String["aa", "bb", "cc"],
)])

# output
);
ClickHouse.insert(sock, "MyTable", [data])
```

```@setup usage
@assert ClickHouse.select(sock, "SELECT * FROM MyTable LIMIT 3") == data
```

## Selecting data

### ... into a dict of `(column, data)` pairs

```jldoctest
select(connect(), "SELECT * FROM MyTable LIMIT 3")

# output

Dict{Symbol,Any} with 3 entries:
:f => Float32[0.0, 2.71828, 3.14159]
:s => ["aa", "bb", "cc"]
:u => UInt64[0x000000000000002a, 0x0000000000000539, 0x000000000000007b]
```@repl usage
ClickHouse.select(sock, "SELECT * FROM MyTable LIMIT 3")
```

### ... into a DataFrame
```jldoctest
select_df(connect(), "SELECT * FROM MyTable LIMIT 3")

# output

3×3 DataFrame
│ Row │ f │ s │ u │
│ │ Float32 │ String │ UInt64 │
├─────┼─────────┼────────┼────────────────────┤
│ 1 │ 0.0 │ aa │ 0x000000000000002a │
│ 2 │ 2.71828 │ bb │ 0x0000000000000539 │
│ 3 │ 3.14159 │ cc │ 0x000000000000007b │

```@repl usage
ClickHouse.select_df(sock, "SELECT * FROM MyTable LIMIT 3")
```

### ... streaming through a channel
```jldoctest
ch = select_channel(connect(), "SELECT * FROM MyTable LIMIT 1")
for block in ch

```@repl usage
for block in ClickHouse.select_channel(sock, "SELECT * FROM MyTable LIMIT 1")
@show block
end

# output

block = Dict{Symbol,Any}(:f => Float32[0.0],:s => ["aa"],:u => UInt64[0x000000000000002a])
```

### ... streaming each block into a callback

This is the fastest way to stream blocks and is used under the hood
to implement all other `select_xyz` implementations.

```jldoctest
select_callback(connect(), "SELECT * FROM MyTable LIMIT 1") do block
```@repl usage
ClickHouse.select_callback(sock, "SELECT * FROM MyTable LIMIT 1") do block
@show block
end
```

# output

block = Dict{Symbol,Any}(:f => Float32[0.0],:s => ["aa"],:u => UInt64[0x000000000000002a])
```
```@setup usage
close(sock)
```
1 change: 1 addition & 0 deletions src/Connect.jl
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ end
)

Return `ClickHouseSock` connected to ClickHouse server with the specified parameters.
The endpoint must use unencrypted native TCP; HTTPS and TLS are not supported.
`compression` selects the format of outgoing data blocks and enables compressed
replies when it is not `COMPRESSION_NONE`. The server chooses the reply format;
the client reads it from each block's header.
Expand Down
4 changes: 4 additions & 0 deletions src/Exceptions.jl
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
#TODO Perhaps we also need stack_trace and nested
"""
An error reported by the ClickHouse server.
Its fields are an integer `code` and strings `name` and `message`.
"""
struct ClickHouseServerException <: Exception
code::Int
name::String
Expand Down
5 changes: 5 additions & 0 deletions src/tcp/ClickHouseSock.jl
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ end
"""is compression enabled in these settings?"""
compression_enabled(settings::CHSettings) = settings.compression != COMPRESSION_NONE

"""
A connection to a ClickHouse server using the native TCP protocol.
Create a connection with [`connect`](@ref), release it with `close`, and
reconnect a closed socket with [`connect!`](@ref).
"""
mutable struct ClickHouseSock
io ::Union{IO, Nothing}
settings ::CHSettings
Expand Down
Loading