Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f8663a4
Parse Snowflake Iceberg version and optional base location
osipovartem Oct 12, 2026
cd984f1
Preserve staged COPY identifier quoting and spans
osipovartem Oct 12, 2026
b08c229
Merge pull request #40 from Embucket/snowflake-copy-ident-spans
osipovartem Oct 12, 2026
56ad61e
Preserve zero-padded COPY staged placeholders
osipovartem Oct 12, 2026
db2c687
Keep COPY placeholder parsing available without std
osipovartem Oct 12, 2026
795ce59
Merge pull request #41 from Embucket/compat-copy-zero-padded-placeholder
osipovartem Oct 12, 2026
6396430
Parse Snowflake ADD COLUMN IF NOT EXISTS
osipovartem Oct 14, 2026
d6c2362
Merge pull request #42 from Embucket/compat-snowflake-add-column-if-n…
osipovartem Oct 14, 2026
33e5ae2
Snowflake: Preserve CREATE DATABASE batch delimiters (#43)
Vedin Nov 6, 2026
9e9559a
Snowflake: Parse typed PUT and GET transfer operands (#44)
Vedin Nov 6, 2026
b1a6571
Snowflake: Accept storage integration after stage endpoint
Vedin Nov 6, 2026
9215f40
Merge pull request #45 from Embucket/oracle/qualify-stage-param-order
Vedin Nov 6, 2026
3e42444
Snowflake: Add typed warehouse statements
Vedin Nov 7, 2026
30e4be5
Merge pull request #46 from Embucket/oracle/qualify-logical-warehouse
Vedin Nov 7, 2026
ab3212c
Snowflake: Test typed NULL table comment syntax
Vedin Nov 7, 2026
3b74008
Snowflake: Preserve typed NULL table comments
Vedin Nov 7, 2026
7d3a763
Snowflake: parse DESC[RIBE] { VIEW | DATABASE | SCHEMA } <name>
sabir-akhadov-localstack Oct 29, 2026
7b4ba1c
Merge pull request #48 from Embucket/backport/describe-object-20261113
osipovartem Nov 13, 2026
f3685b3
Snowflake: preserve CREATE SCHEMA literal options
osipovartem Nov 13, 2026
55cf93f
Merge pull request #49 from Embucket/feat/snowflake-schema-options-20…
osipovartem Nov 13, 2026
d5b2f70
Support Snowflake ALTER SCHEMA property syntax
osipovartem Nov 13, 2026
3cc4ae4
Merge pull request #50 from Embucket/feat/snowflake-alter-schema-opti…
osipovartem Nov 13, 2026
7fdc000
Parse native Snowflake ALTER DATABASE property operations
osipovartem Nov 13, 2026
874cfbb
feat: preserve Snowflake CREATE SCHEMA modifiers in native AST
osipovartem Nov 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/license.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,6 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: "3.12"
- name: Audit licenses
run: ./dev/release/run-rat.sh .
run: bash -e ./dev/release/run-rat.sh .
29 changes: 29 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,35 @@ snowflake().one_statement_parses_to(
```

## Analyzing Parsing Issues

Warehouse commands use Snowflake-only typed syntax in
`ast/helpers/stmt_warehouse.rs`, dispatched through the existing Snowflake dialect
override. Keep account-object names distinct from table relations. Preserve ordered
primitive properties, including duplicate or semantically invalid values, for the
execution consumer to validate. Never evaluate properties, normalize account identity,
consume statement delimiters inside option loops, or infer runtime support from an AST.
Run the required fork checks below before committing changes to this syntax.

Namespace DESCRIBE backports retain this fork's existing VIEW representation
(`ExplainTable::has_view_keyword`), stage/file-format/sequence syntax and statement
boundaries. `DescribeObject` admits DATABASE/SCHEMA only; account namespaces must
not acquire table-relation visitor annotations. Parser tests do not establish
DataFusion or Rustice runtime support for the additive AST variant.

Snowflake CREATE TABLE's dedicated COMMENT option admits explicit `= NULL` as
`SqlOption::Comment(CommentDef::NullWithEq)` in plain table options. Keep existing
string variants and the shared `parse_comment_value` strict: column, stage, file
format, view and connector comments do not gain NULL syntax. Do not collapse NULL
into an empty string or absent clause, infer no-`=` NULL syntax, or rewrite SQL.
The additive public enum variant requires an exhaustive-match/serde consumer audit
and a coherent parser/DataFusion/engine pin qualification before runtime claims.

For Snowflake `PUT`/`GET`, keep opaque location lexing confined to the two operands
at actual statement start. Preserve ordinary comments and identifiers/functions
named `PUT`/`GET`. Keep stage `ObjectName` quoting separate from the exact payload
prefix, avoid table-relation visitor annotations for named stages, and retain
bindings as normal expression placeholders. The parser must not resolve paths,
perform storage IO, evaluate options, or enforce an engine's transfer policy.
You can try to simplify the SQL statement to identify the root cause of the parsing issue. This may involve removing certain clauses or components of the SQL statement to see if it can be parsed successfully. Additionally, you can compare the problematic SQL statement with similar statements that are parsed correctly to identify any differences that may be causing the issue.

## Pre Commit Checks
Expand Down
57 changes: 56 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,26 @@ This crate contains a lexer and parser for SQL that conforms with the
is used as a foundation for SQL query engines, vendor-specific
parsers, and various SQL analysis.

Snowflake warehouse parsing uses typed CREATE, ALTER, SHOW,
DESCRIBE and DROP nodes in `ast::helpers::stmt_warehouse`; USE keeps its existing
AST. It retains quoted/multipart account names and ordered primitive options for
later semantic validation. Parsing does not implement warehouses, authorize an
operation, evaluate properties, or establish wire metadata/error parity. Tags,
CREATE OR ALTER, adaptive/interactive control, ABORT ALL QUERIES and SHOW privilege
filters remain outside this selected grammar.

The Snowflake table-comment source preparation adds explicit `COMMENT = NULL` as
`SqlOption::Comment(CommentDef::NullWithEq)` within `CreateTableOptions::Plain`.
An absent clause, `COMMENT = ''`, `COMMENT = 'NULL'` and typed NULL remain distinct.
Existing string and dollar-quoted comments keep their string AST variants. Only
the dedicated CREATE TABLE arm admits NULL; shared column/stage/file-format/view
and connector string parsers remain strict, as does table `COMMENT NULL` without
`=`. The selected Snowflake 10.40.102 reference is a temporary CREATE table
(query `01c83e8c-0209-ab5f-0004-d66e05c171ae`), without a persistence/runtime claim.
This preparation is unqualified until the fork checks run. The additive public
enum variant can break external exhaustive matches or old serde readers; existing
string constructors/representations stay intact, without a binary ABI guarantee.

## Example

To parse a simple `SELECT` statement:
Expand Down Expand Up @@ -78,7 +98,42 @@ This crate avoids semantic analysis because it varies drastically
between dialects and implementations. If you want to do semantic
analysis, feel free to use this project as a base.

## Preserves Syntax Round Trip
## Snowflake transfer operands

`SnowflakeDialect` represents `PUT` and `GET` with dedicated statement nodes and
typed local URI, stage, and binding operands. A named stage retains its real
`ObjectName` identifiers and a separate exact payload prefix. User and table
stages remain distinct AST forms. Stage operands are not annotated as table
relations for generic relation discovery.

Only the first two operands after a statement-start transfer keyword use opaque
location tokenization. This keeps `file://`, `/*` globs, `--` filenames, `?`, and
`#` intact without changing ordinary SQL comments or identifiers/functions named
`PUT` and `GET`. Single-quoted transfer operands retain literal backslashes and
use doubled quotes for a literal quote; ordinary SQL string escape rules outside
these operands are unchanged. Whole-stage single quotes are a local grammar form,
not independent Snowflake reference evidence. Double-quoted local URIs and
`IDENTIFIER(...)` transfer operands are outside this grammar.

Options retain their original order, duplicates, and typed `Expr` values, including
signed numeric spelling. Optional numeric spelling metadata retains forms such as
`001` even when the `bigdecimal` feature normalizes the typed number; rendering
ignores stale spelling after a visitor changes that numeric value. Location
bindings use normal placeholder expressions so
expression visitors can apply the owning engine's existing binding accounting.
`dialect::snowflake::parse_file_transfer_stage_location` converts an exact bound
stage value into a typed target without manufacturing a SQL statement.

The parser performs no storage IO, option evaluation, range checking, async/batch
admission, or transfer execution. The execution layer must validate primitive
option shapes and supported stage kinds before planning/evaluating expressions
or submitting work. Accepting syntax does not claim a transfer transport or a
complete Snowflake workflow. In particular, path traversal/backslash admission,
`PARALLEL` range errors, duplicate/unknown options, and unsupported stage kinds
remain semantic responsibilities. These additions require owning qualification;
source tests alone do not establish runtime or reference parity.

## Preserves Syntax Round Trip

This crate allows users to recover the original SQL text (with comments removed,
normalized whitespace and keyword capitalization), which is useful for tools
Expand Down
4 changes: 2 additions & 2 deletions examples/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ $ cargo run --example cli - [--dialectname]
.expect("failed to read from stdin");
String::from_utf8(buf).expect("stdin content wasn't valid utf8")
} else {
println!("Parsing from file '{}' using {:?}", &filename, dialect);
println!("Parsing from file '{}' using {:?}", filename, dialect);
fs::read_to_string(&filename)
.unwrap_or_else(|_| panic!("Unable to read the file {}", &filename))
.unwrap_or_else(|_| panic!("Unable to read the file {}", filename))
};
let without_bom = if contents.chars().next().unwrap() as u64 != 0xfeff {
contents.as_str()
Expand Down
81 changes: 80 additions & 1 deletion src/ast/ddl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3024,6 +3024,9 @@ pub struct CreateTable {
/// Snowflake "BASE_LOCATION" clause for Iceberg tables
/// <https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table>
pub base_location: Option<String>,
/// Snowflake "ICEBERG_VERSION" clause for Iceberg tables.
/// <https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake>
pub iceberg_version: Option<u64>,
/// Snowflake "CATALOG" clause for Iceberg tables
/// <https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table>
pub catalog: Option<String>,
Expand Down Expand Up @@ -3265,6 +3268,9 @@ impl fmt::Display for CreateTable {
if let Some(base_location) = self.base_location.as_ref() {
write!(f, " BASE_LOCATION='{base_location}'")?;
}
if let Some(iceberg_version) = self.iceberg_version {
write!(f, " ICEBERG_VERSION={iceberg_version}")?;
}
}

if let Some(catalog_sync) = self.catalog_sync.as_ref() {
Expand Down Expand Up @@ -3788,6 +3794,63 @@ impl fmt::Display for CreateConnector {
}
}

/// A native Snowflake `ALTER DATABASE` property operation.
#[derive(Debug, Clone, PartialEq, PartialOrd, Eq, Ord, Hash)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[cfg_attr(feature = "visitor", derive(Visit, VisitMut))]
pub enum AlterDatabaseOperation {
/// Set primitive database properties without an `OPTIONS (...)` wrapper.
SetOptions {
/// Ordered assignments, including duplicates; values are not semantically validated.
options: Vec<SqlOption>,
},
/// Reset database properties to their defaults.
UnsetOptions {
/// Ordered property names, including duplicates.
options: Vec<Ident>,
},
}

impl fmt::Display for AlterDatabaseOperation {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Self::SetOptions { options } => write!(f, "SET {}", display_separated(options, " ")),
Self::UnsetOptions { options } => {
write!(f, "UNSET {}", display_comma_separated(options))
}
}
}
}

/// A Snowflake `ALTER DATABASE` (`Statement::AlterDatabase`) statement.
///
/// See [Snowflake](https://docs.snowflake.com/en/sql-reference/sql/alter-database).
#[derive(Debug, Clone, PartialEq, PartialOrd, Eq, Ord, Hash)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[cfg_attr(feature = "visitor", derive(Visit, VisitMut))]
pub struct AlterDatabase {
/// The account-level database name, not a table relation.
pub name: ObjectName,
/// Whether `IF EXISTS` was specified.
pub if_exists: bool,
/// The property operations to perform on the database.
pub operations: Vec<AlterDatabaseOperation>,
}

impl fmt::Display for AlterDatabase {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "ALTER DATABASE ")?;
if self.if_exists {
write!(f, "IF EXISTS ")?;
}
write!(f, "{}", self.name)?;
for operation in &self.operations {
write!(f, " {operation}")?;
}
Ok(())
}
}

/// An `ALTER SCHEMA` (`Statement::AlterSchema`) operation.
///
/// See [BigQuery](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#alter_schema_collate_statement)
Expand Down Expand Up @@ -3818,6 +3881,16 @@ pub enum AlterSchemaOperation {
/// The options to set.
options: Vec<SqlOption>,
},
/// Set primitive schema properties without an `OPTIONS (...)` wrapper (Snowflake).
SetOptions {
/// Ordered assignments, including duplicates; values are not semantically validated.
options: Vec<SqlOption>,
},
/// Reset schema properties to their defaults (Snowflake).
UnsetOptions {
/// Ordered property names, including duplicates.
options: Vec<Ident>,
},
/// Rename the schema.
Rename {
/// The new name for the schema.
Expand Down Expand Up @@ -3847,6 +3920,12 @@ impl fmt::Display for AlterSchemaOperation {
AlterSchemaOperation::SetOptionsParens { options } => {
write!(f, "SET OPTIONS ({})", display_comma_separated(options))
}
AlterSchemaOperation::SetOptions { options } => {
write!(f, "SET {}", display_separated(options, " "))
}
AlterSchemaOperation::UnsetOptions { options } => {
write!(f, "UNSET {}", display_comma_separated(options))
}
AlterSchemaOperation::Rename { name } => write!(f, "RENAME TO {name}"),
AlterSchemaOperation::OwnerTo { owner } => write!(f, "OWNER TO {owner}"),
}
Expand Down Expand Up @@ -4698,7 +4777,7 @@ impl fmt::Display for AlterTable {
if self.only {
write!(f, "ONLY ")?;
}
write!(f, "{} ", &self.name)?;
write!(f, "{} ", self.name)?;
if let Some(cluster) = &self.on_cluster {
write!(f, "ON CLUSTER {cluster} ")?;
}
Expand Down
2 changes: 2 additions & 0 deletions src/ast/helpers/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ pub mod stmt_create_database;
pub mod stmt_create_table;
/// Helpers for data loading/unloading related statements (stages, PUT, COPY INTO).
pub mod stmt_data_loading;
/// Snowflake warehouse lifecycle syntax, without execution or account semantics.
pub mod stmt_warehouse;
5 changes: 5 additions & 0 deletions src/ast/helpers/stmt_create_table.rs
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,8 @@ pub struct CreateTableBuilder {
pub with_tags: Option<Vec<Tag>>,
/// Optional base location for staged data.
pub base_location: Option<String>,
/// Optional Iceberg format version.
pub iceberg_version: Option<u64>,
/// Optional external volume identifier.
pub external_volume: Option<String>,
/// Optional catalog name.
Expand Down Expand Up @@ -234,6 +236,7 @@ impl CreateTableBuilder {
with_storage_lifecycle_policy: None,
with_tags: None,
base_location: None,
iceberg_version: None,
external_volume: None,
catalog: None,
catalog_sync: None,
Expand Down Expand Up @@ -604,6 +607,7 @@ impl CreateTableBuilder {
with_storage_lifecycle_policy: self.with_storage_lifecycle_policy,
with_tags: self.with_tags,
base_location: self.base_location,
iceberg_version: self.iceberg_version,
external_volume: self.external_volume,
catalog: self.catalog,
catalog_sync: self.catalog_sync,
Expand Down Expand Up @@ -685,6 +689,7 @@ impl From<CreateTable> for CreateTableBuilder {
with_storage_lifecycle_policy: table.with_storage_lifecycle_policy,
with_tags: table.with_tags,
base_location: table.base_location,
iceberg_version: table.iceberg_version,
external_volume: table.external_volume,
catalog: table.catalog,
catalog_sync: table.catalog_sync,
Expand Down
Loading
Loading