Duckdb: First Release

[Mytherin]

Required

• [x] Documentation and Website
• [ ] More examples
• [x] Download button that actually works on Website
• [ ] Storage (#29, #77, #139, #48, #49, #109)
• [x] Timestamp (#71)
• [x] NOW, CURRENT_DATE functions (#149)
• [x] TPC-DS Correct (#85)
• [x] 32-bit support (on Windows as well) (#172)
• [x] Packaging (#174)
• [x] R/Python Clients (Python: #101, #102, #103, R: #120)
• [x] Shell on Windows/shell fixes

• [x] Prevent dropping of "main" schema (#154)

  Preferable

• [x] Transient ART index (#62)

• [ ] Set default schema that is used at runtime (#155)
• [ ] Add compatibility with several MySQL functions (#156)
• [ ] LIKE fixes (#157, #158, #163, #164)
• [x] Add type to result set and fix rendering in shell to only render results of SELECT statements (#159)
• [x] CONCAT with multiple parameters (#167)

• [ ] Warnings in connection (#170)

  Possible, but not critical

• [x] Blob type (#72)

• [x] Temporary tables (#58)
• [ ] CTEs with other statement types (#160)
• [x] DISTINCT ON (#162)
• [ ] Support explicit schema in column ref (#146)

[Mytherin]

I will implement the storage in the following steps. Each of these steps will involve a merge into master to avoid having a big storage merge at the end.

• [x] Change checkpointing to single file storage layout as described in the storage presentation, this also includes checksumming (#29) and adds locking as described in #77
• [x] Change WAL to use FileSystem API (fixes #139), also only checkpoint when the WAL reaches sufficient size
• [x] Change loading on startup so the entire database is not loaded into memory immediately. This basically creates a non-evicting buffer manager. We also need to consider the UniqueIndex here; it should only be created on INSERT or UPDATE
• [x] Add evicting to the BufferManager together where pages are evicted based on some algorithm. We still only consider the base tables in the BufferManager in this phase. (#48)
• [ ] Add incremental checkpointing so the entire database does not need to be rebuild every time the checkpoint process is run. Also checkpoint whenever the WAL reaches a certain size instead of only on startup. (#49)
• [ ] Also incorporate temporary structures into the BufferManager, primarily the ChunkCollection, Join HashTable and Aggregate HashTable.
• [ ] Add user-triggered vacuum operation that compacts blocks with empty space into fewer blocks that are full (#109)

[hannesmuehleisen]

Regarding the download button, what should happen when one pushes it? go to releases page a la https://www.sqlite.org/download.html? ?

[Mytherin]

I think node.js gets the download button right: https://nodejs.org/en/

i.e. automatically switch the download based on detected OS.

It's a bit more difficult for us because we also have different language versions. We could have something similar to the Spark documentation where you switch based on language. We can have some different code snippets for installing it for Python/R (e.g. pip install duckdb) and some tiny usage examples on the main page.

[hannesmuehleisen]

i think target language switch might be nicest, so you say "C/C++/Python/R" and we show you the code snippet or command to install and some very basic usage