tencent cloud

Data Lake Compute

CREATE TABLE

Download
Focus Mode
Font Size
Last updated: 2026-05-28 10:25:04
AI-Translated

Description

Supported Kernels: Presto, SparkSQL.
Applicable Table Range: Native Iceberg Tables, External Tables.
Purpose: Create a table with some attributes, supporting the CREATE TABLE AS syntax.
Table Creation Storage Path: The storage path for table creation supports specifying to a COS directory, but not to a file.

External Table Syntax

Syntax

CREATE TABLE [ IF NOT EXISTS ] table_identifier
( col_name[:] col_type [ COMMENT col_comment ], ... )
USING data_source
[ COMMENT table_comment ]
[ OPTIONS ( 'key1'='value1', 'key2'='value2' )]
[ PARTITIONED BY ( col_name1, transform(col_name2), ... ) ]
[ LOCATION path ]
[ TBLPROPERTIES ( property_name=property_value, ... ) ]

Parameter

USING data_source: Input type of data when creating a table, currently includes: CSV, ORC, PARQUET, ICEBERG, etc. table_identifier: Specify the table name, supporting three-part structure, e.g., catalog.database.table. COMMENT: Description of the table.
OPTIONS: Extra parameters supported by USING data_source, used for parameter injection during storage. PARTITIONED BY: Create partitions based on specified columns. LOCATION path: Storage path of the data table. TBLPROPERTIES: A set of k-v values used to specify parameters of the table.

Detailed explanation of USING and OPTIONS parameters

USING CSV
USING ORC
USING PARQUET
The following configurations are supported for CSV data tables.
Keys supported by OPTIONS
Default values for the key corresponding to the value
Meaning
sep or delimiter
,
The separator between columns when storing CSV, default is English comma
mode
PERMISSIVE
Definition: The processing mode when data conversion does not meet expectations.
PERMISSIVE: Loose Mode, default, tries to convert a certain row of data, for example, if a certain row has extra columns, it will automatically select only the required columns
DROPMALFORMED: Discard data that does not meet expectations, for example, if a row has extra columns, that row will be discarded
FAILFAST: Strictly requires CSV format; if a row does not meet expectations, it fails immediately, for example, in the case of extra columns.
encoding or charset
UTF-8
String encoding format.
Examples: UTF-8, US-ASCII, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-16
quote
\\"
Are the quotation marks single or double? Pay attention to the use of escape characters
escape
\\\\
Escape Character, pay attention to the use of escape characters
charToEscapeQuoteEscaping
-
Characters inside quotation marks need escaping
comment
\\u0000
Remark information
header
false
Header present
inferSchema
false
Infer the type of each column. If not inferred, each column is treated as a string
ignoreLeadingWhiteSpace
Read: false
Write: true
Ignore leading empty strings
ignoreTrailingWhiteSpace
Read: false
Write: true
Ignore trailing empty strings
columnNameOfCorruptRecord
_corrupt_record
Column name of non-convertible columns, affected by spark.sql.columnNameOfCorruptRecord, primarily based on table configuration
nullValue
-
Storage format of null, default is an empty string, written as emptyValue
nanValue
NaN
Storage format of non-numeric values
positiveInf
Inf
Storage format of positive infinity
negativeInf
-Inf
Storage format of negative infinity
compression or codec
-
Class name of the compression algorithm, default is no compression. Short names can be used: bzip2, deflate, gzip, lz4, snappy
timeZone
System default time zone
Default time zone, this parameter is influenced by spark.sql.session.timeZone. For example, Asia/Shanghai. Primarily follows table configuration
locale
en-US
Language
dateFormat
yyyy-MM-dd
Default date format
timestampFormat
yyyy-MM-dd'T'HH:mm:ss.SSSXXX
Default time format, in non-LEGACY mode it is yyyy-MM-dd'T'HH:mm:ss[.SSS][XXX]
multiLine
false
Allow multiple lines
maxColumns
20480
Maximum number of columns
maxCharsPerColumn
-1
Maximum characters per column, -1 means no limit
escapeQuotes
true
Escape quotation marks
quoteAll
quoteAll
Enclose the entire content in quotation marks when writing
samplingRatio
1.0
Sampling rate
enforceSchema
true
Force the use of specified schema for reading, ignoring the table header's definition
emptyValue
Read:
Write: ""
Read-Write format for null values
lineSep
-
Line break
inputBufferSize
-
Buffer size during read, this parameter can be influenced by spark.sql.csv.parser.inputBufferSize. Primarily follows table configuration
unescapedQuoteHandling
STOP_AT_DELIMITER
Handling strategy when non-escaping quotes are found.
STOP_AT_DELIMITER: Stop at delimiter
BACK_TO_DELIMITER: Rewind to delimiter
STOP_AT_CLOSING_QUOTE: Stop at the next quote
SKIP_VALUE: Skip this column of data
RAISE_ERROR: Raise an error

Supported configurations for ORC data tables are as follows:
Keys supported by OPTIONS
Default values for the key corresponding to the value
Meaning
compression or orc.compress
snappy
Compression Algorithm, supports abbreviations snappy/zlib/lzo/lz3/zstd. This parameter is affected by spark.sql.orc.compression.codec but gives priority to table parameters
mergeSchema
false
Merge schema. This parameter is affected by spark.sql.orc.mergeSchema but gives priority to table parameters
If using HiveRead and HiveWriter (with spark.sql.hive.convertMetastoreOrc=false) for read and write operations, OPTIONS can also support native Orc configurations. For details, please refer to LanguageManual ORC.
Most parameters related to PARQUET data tables can be configured through Spark conf. It is also recommended to configure from Spark conf. Options supported are as follows:
Keys supported by OPTIONS
Default values for the key corresponding to the value
Meaning
compression or parquet.compression
snappy
Compression Algorithm, defaults to snappy. Affected by the parameter spark.sql.parquet.compression.codec but gives priority to table parameters.
mergeSchema
false
Whether to merge schema, affected by the parameter spark.sql.parquet.mergeSchema but gives priority to table parameters.
datetimeRebaseMode
EXCEPTION
The conversion strategy for dates when writing parquet files. LEGACY mode converts dates using the Gregorian calendar, CORRECTED mode does not convert dates to the Gregorian calendar, EXCEPTION mode throws an error when encountering dates of different formats. Affected by the parameter spark.sql.parquet.datetimeRebaseModeInRead but gives priority to table parameters.
int96RebaseMode
EXCEPTION
The conversion strategy for time when reading parquet files. LEGACY mode converts time using the Gregorian calendar, CORRECTED mode does not convert time, and EXCEPTION mode throws an error when encountering different time formats. Affected by the parameter spark.sql.parquet.int96RebaseModeInRead but gives priority to table parameters.
If using HiveRead and HiveWriter (with spark.sql.hive.convertMetastoreParquet=false) for read and write operations, OPTIONS can also support native Parquet configurations. Refer to: Hadoop integration.

Sample code

CREATE TABLE dempts(
id bigint COMMENT 'id number',
num int,
eno float,
dno double,
cno decimal(9,3),
flag boolean,
data string,
ts_year timestamp,
date_month date,
bno binary,
point struct<x: double, y: double>,
points array<struct<x: double, y: double>>,
pointmaps map<struct<x: int>, struct<a: int>>
)
USING iceberg
COMMENT 'table documentation'
PARTITIONED BY (bucket(16,id), years(ts_year), months(date_month), identity(bno), bucket(3,num), truncate(10,data))
LOCATION '/warehouse/db_001/dempts'
TBLPROPERTIES ('write.format.default'='orc');

FAQs

In the keywords when using CREATE_TABLE, there are differences between Spark's USING and Hive's STORED AS, which may lead to file format and reading not meeting expectations after table creation. Here's a special note:
USING DATA_SOURCE: Spark syntax, this keyword indicates the data source format to be used when creating a table, directly affecting the file format and reading method under the table's location. Values can include CSV, TXT, Iceberg, Parquet, Orc, etc.
STORED AS FILE_FORMAT: Hive syntax, this keyword is used to create a HIVE format table, indicating the format of data files stored in the table. Values can include TXT, Parquet, Orc, etc. It is not recommended to use this syntax, as it may cause Spark's native reader/writer to be unsupported, for example, not supporting CSV.

Native Table Iceberg Syntax

Caution
This syntax only supports creating native tables.

Syntax

CREATE TABLE [ IF NOT EXISTS ] table_identifier
( col_name[:] col_type [ COMMENT col_comment ], ... )
[ COMMENT table_comment ]
[ PARTITIONED BY ( col_name1, transform(col_name2), ... ) ]

Parameter

table_identifier: Supports three-part naming, catalog.db.name Schemas and Data Types
col_type
: primitive_type
  | nested_type

primitive_type
: boolean
| int/integer
| long/bigint
| float
| double
| decimal(p,s), p = maximum digits, s = maximum decimal places, s<=p<=38
| date
| timestamp, timestamp with timezone, does not support time and without timezone
| string, also corresponds to Iceberg uuid type
| binary, also corresponds to Iceberg fixed type

nested_type
: struct
| list
| map
Partition Transforms
transform
: identity, supports any type, DLC does not support this conversion
| bucket[N], hash mod N bucketing, supports col_type: int, long, decimal, date, timestamp, string, binary
| truncate[L], L-truncation bucketing, supports col_type: int, long, decimal, string
| years, Year, supports col_type: date, timestamp
| months, Month, supports col_type: date, timestamp
| days/date, Date, supports col_type: date, timestamp
| hours/date_hour, Hour, supports col_type: timestamp

Sample code

CREATE TABLE dempts(
id bigint COMMENT 'id number',
num int,
eno float,
dno double,
cno decimal(9,3),
flag boolean,
data string,
ts_year timestamp,
date_month date,
bno binary,
point struct<x: double, y: double>,
points array<struct<x: double, y: double>>,
pointmaps map<struct<x: int>, struct<a: int>>
)
COMMENT 'table documentation'
PARTITIONED BY (bucket(16,id), years(ts_year), months(date_month), identity(bno), bucket(3,num), truncate(10,data));


Help and Support

Was this page helpful?

Help us improve! Rate your documentation experience in 5 mins.

Feedback