Skip to content

API Coverage

This page shows which APIs are currently re-implemented by pyspark-dubber. This list is not exhaustive, showing mostly public functions and DataFrame APIs, however some additional APIs and magic methods are also implemented.

In addition to that, certain pyspark APIs are partially implemented, for example not all parameters or parameter types are supported. In spite of that, they are listed as implemented in the tables below, with notes in case of partial implementation.

The overall approximate API coverage (with the caveats above) is 46.3%. We prioritize implementing commonly used functions, as pyspark has many esoteric APIs.

SparkSession (3/22 = 14%)

API Implemented Notes
SparkSession.Builder
SparkSession.active
SparkSession.addArtifact
SparkSession.addArtifacts
SparkSession.addTag
SparkSession.clearProgressHandlers
SparkSession.clearTags
SparkSession.copyFromLocalToFs
SparkSession.createDataFrame Generally createDataFrame is a complex method, so certain edge cases are not handled correctly. Some notable incompatibilities with pyspark:
SparkSession.getActiveSession
SparkSession.getTags
SparkSession.interruptAll
SparkSession.interruptOperation
SparkSession.interruptTag
SparkSession.newSession
SparkSession.range
SparkSession.registerProgressHandler
SparkSession.removeProgressHandler
SparkSession.removeTag
SparkSession.sql
SparkSession.stop
SparkSession.table

SparkSession.builder (3/7 = 43%)

API Implemented Notes
Builder.appName
Builder.config
Builder.create
Builder.enableHiveSupport
Builder.getOrCreate
Builder.master
Builder.remote

Input Formats (2/13 = 15%)

API Implemented Notes
DataFrameReader.csv
DataFrameReader.format
DataFrameReader.jdbc
DataFrameReader.json Most parameters are accepted but completely ignored.
DataFrameReader.load
DataFrameReader.option
DataFrameReader.options
DataFrameReader.orc
DataFrameReader.parquet
DataFrameReader.schema
DataFrameReader.table
DataFrameReader.text
DataFrameReader.xml

Output Formats (4/18 = 22%)

API Implemented Notes
DataFrameWriter.bucketBy
DataFrameWriter.clusterBy
DataFrameWriter.csv Most parameters are unsupported, the writing of files cannot be reproduced 1:1because it depends on spark internals such as partitions.
DataFrameWriter.format
DataFrameWriter.insertInto
DataFrameWriter.jdbc
DataFrameWriter.json
DataFrameWriter.mode
DataFrameWriter.option
DataFrameWriter.options
DataFrameWriter.orc
DataFrameWriter.parquet Most parameters are unsupported, the writing of files cannot be reproduced 1:1because it depends on spark internals such as partitions.
DataFrameWriter.partitionBy
DataFrameWriter.save
DataFrameWriter.saveAsTable
DataFrameWriter.sortBy
DataFrameWriter.text
DataFrameWriter.xml

DataFrame (49/102 = 48%)

API Implemented Notes
DataFrame.agg
DataFrame.alias
DataFrame.approxQuantile
DataFrame.asTable
DataFrame.cache
DataFrame.checkpoint
DataFrame.coalesce
DataFrame.colRegex
DataFrame.collect
DataFrame.corr
DataFrame.count
DataFrame.cov
DataFrame.createGlobalTempView
DataFrame.createOrReplaceGlobalTempView
DataFrame.createOrReplaceTempView
DataFrame.createTempView
DataFrame.crossJoin pyspark allows duplicate column names, and by default does not prefix/suffix the columns of the other dataframe at all. Our backend (ibis) currently does not support duplicate column names, so this function suffixes all columns on other with '_right'.
DataFrame.crosstab
DataFrame.cube
DataFrame.describe
DataFrame.distinct
DataFrame.drop Our backend (ibis) does not support duplicate column names like pyspark, therefore this function does not support dropping columns with the same name. You cannot anyway currently create such dataframes using pyspark-dubber.
DataFrame.dropDuplicates
DataFrame.dropDuplicatesWithinWatermark
DataFrame.drop_duplicates
DataFrame.dropna The thresh parameter is not honored.
DataFrame.exceptAll
DataFrame.exists
DataFrame.explain
DataFrame.fillna
DataFrame.filter
DataFrame.first
DataFrame.foreach
DataFrame.foreachPartition
DataFrame.freqItems
DataFrame.groupBy Currently only column names are supported for grouping, column expressions are not supported.
DataFrame.groupby Currently only column names are supported for grouping, column expressions are not supported.
DataFrame.groupingSets
DataFrame.head
DataFrame.hint
DataFrame.inputFiles
DataFrame.intersect
DataFrame.intersectAll
DataFrame.isEmpty
DataFrame.isLocal
DataFrame.join
DataFrame.lateralJoin
DataFrame.limit
DataFrame.localCheckpoint
DataFrame.mapInArrow
DataFrame.mapInPandas
DataFrame.melt
DataFrame.mergeInto
DataFrame.metadataColumn
DataFrame.observe
DataFrame.offset
DataFrame.orderBy Sorting by column ordinals (which are 1-based, not 0-based) is not supported yet. Additionally, this function still needs better testing around edge cases, when sorting with complex column expressions which include sorting.
DataFrame.pandas_api
DataFrame.persist
DataFrame.printSchema The level parameter is not honored.
DataFrame.randomSplit
DataFrame.registerTempTable
DataFrame.repartition
DataFrame.repartitionById
DataFrame.repartitionByRange
DataFrame.replace
DataFrame.rollup
DataFrame.sameSemantics
DataFrame.sample
DataFrame.sampleBy
DataFrame.scalar
DataFrame.select
DataFrame.selectExpr
DataFrame.semanticHash
DataFrame.show The truncate and vertical parameters are not honored. Additionally, the output is not printed justified exactly as pyspark as of the current version.
DataFrame.sort Sorting by column ordinals (which are 1-based, not 0-based) is not supported yet. Additionally, this function still needs better testing around edge cases, when sorting with complex column expressions which include sorting.
DataFrame.sortWithinPartitions Sorting by column ordinals (which are 1-based, not 0-based) is not supported yet. Additionally, this function still needs better testing around edge cases, when sorting with complex column expressions which include sorting.
DataFrame.subtract
DataFrame.summary
DataFrame.tail
DataFrame.take
DataFrame.to
DataFrame.toArrow
DataFrame.toDF
DataFrame.toJSON
DataFrame.toLocalIterator
DataFrame.toPandas
DataFrame.transform
DataFrame.transpose
DataFrame.union
DataFrame.unionAll
DataFrame.unionByName
DataFrame.unpersist
DataFrame.unpivot
DataFrame.where
DataFrame.withColumn
DataFrame.withColumnRenamed
DataFrame.withColumns
DataFrame.withColumnsRenamed
DataFrame.withMetadata
DataFrame.withWatermark
DataFrame.writeTo

GroupBy (7/15 = 47%)

API Implemented Notes
GroupedData.agg
GroupedData.apply
GroupedData.applyInArrow
GroupedData.applyInPandas
GroupedData.applyInPandasWithState
GroupedData.avg
GroupedData.cogroup
GroupedData.count
GroupedData.max
GroupedData.mean
GroupedData.min
GroupedData.pivot
GroupedData.sum
GroupedData.transformWithState
GroupedData.transformWithStateInPandas

Column (26/36 = 72%)

API Implemented Notes
Column.alias
Column.asc
Column.asc_nulls_first
Column.asc_nulls_last
Column.astype
Column.between
Column.bitwiseAND
Column.bitwiseOR
Column.bitwiseXOR
Column.cast
Column.contains
Column.desc
Column.desc_nulls_first
Column.desc_nulls_last
Column.dropFields
Column.endswith
Column.eqNullSafe
Column.getField
Column.getItem
Column.ilike
Column.isNaN
Column.isNotNull
Column.isNull
Column.isin
Column.like
Column.name
Column.otherwise
Column.outer
Column.over
Column.rlike
Column.startswith
Column.substr
Column.transform
Column.try_cast
Column.when
Column.withField

Functions (227/503 = 45%)

API Implemented Notes
pyspark.sql.functions.abs
pyspark.sql.functions.acos
pyspark.sql.functions.acosh
pyspark.sql.functions.add_months
pyspark.sql.functions.aes_decrypt
pyspark.sql.functions.aes_encrypt
pyspark.sql.functions.aggregate
pyspark.sql.functions.any_value
pyspark.sql.functions.approxCountDistinct
pyspark.sql.functions.approx_count_distinct
pyspark.sql.functions.approx_percentile The accuracy argument is not honored.
pyspark.sql.functions.array
pyspark.sql.functions.array_agg
pyspark.sql.functions.array_append
pyspark.sql.functions.array_compact
pyspark.sql.functions.array_contains
pyspark.sql.functions.array_distinct
pyspark.sql.functions.array_except
pyspark.sql.functions.array_insert
pyspark.sql.functions.array_intersect
pyspark.sql.functions.array_join null_replacement is not natively in ibis
pyspark.sql.functions.array_max
pyspark.sql.functions.array_min
pyspark.sql.functions.array_position
pyspark.sql.functions.array_prepend
pyspark.sql.functions.array_remove
pyspark.sql.functions.array_repeat
pyspark.sql.functions.array_size
pyspark.sql.functions.array_sort comparator parameter is not supported
pyspark.sql.functions.array_union
pyspark.sql.functions.arrays_overlap
pyspark.sql.functions.arrays_zip
pyspark.sql.functions.arrow_udtf
pyspark.sql.functions.asc
pyspark.sql.functions.asc_nulls_first
pyspark.sql.functions.asc_nulls_last
pyspark.sql.functions.ascii
pyspark.sql.functions.asin
pyspark.sql.functions.asinh
pyspark.sql.functions.assert_true
pyspark.sql.functions.atan
pyspark.sql.functions.atan2
pyspark.sql.functions.atanh
pyspark.sql.functions.avg
pyspark.sql.functions.base64
pyspark.sql.functions.bin
pyspark.sql.functions.bit_and
pyspark.sql.functions.bit_count
pyspark.sql.functions.bit_get
pyspark.sql.functions.bit_length
pyspark.sql.functions.bit_or
pyspark.sql.functions.bit_xor
pyspark.sql.functions.bitmap_and_agg
pyspark.sql.functions.bitmap_bit_position
pyspark.sql.functions.bitmap_bucket_number
pyspark.sql.functions.bitmap_construct_agg
pyspark.sql.functions.bitmap_count
pyspark.sql.functions.bitmap_or_agg
pyspark.sql.functions.bitwiseNOT
pyspark.sql.functions.bitwise_not
pyspark.sql.functions.bool_and
pyspark.sql.functions.bool_or
pyspark.sql.functions.broadcast
pyspark.sql.functions.bround
pyspark.sql.functions.btrim
pyspark.sql.functions.bucket
pyspark.sql.functions.call_function
pyspark.sql.functions.call_udf
pyspark.sql.functions.cardinality
pyspark.sql.functions.cbrt
pyspark.sql.functions.ceil
pyspark.sql.functions.ceiling
pyspark.sql.functions.char
pyspark.sql.functions.char_length
pyspark.sql.functions.character_length
pyspark.sql.functions.chr
pyspark.sql.functions.coalesce
pyspark.sql.functions.col
pyspark.sql.functions.collate
pyspark.sql.functions.collation
pyspark.sql.functions.collect_list
pyspark.sql.functions.collect_set
pyspark.sql.functions.column
pyspark.sql.functions.concat
pyspark.sql.functions.concat_ws
pyspark.sql.functions.contains
pyspark.sql.functions.conv
pyspark.sql.functions.convert_timezone
pyspark.sql.functions.corr
pyspark.sql.functions.cos
pyspark.sql.functions.cosh
pyspark.sql.functions.cot
pyspark.sql.functions.count
pyspark.sql.functions.countDistinct
pyspark.sql.functions.count_distinct
pyspark.sql.functions.count_if
pyspark.sql.functions.count_min_sketch
pyspark.sql.functions.covar_pop
pyspark.sql.functions.covar_samp
pyspark.sql.functions.crc32
pyspark.sql.functions.create_map
pyspark.sql.functions.csc
pyspark.sql.functions.cume_dist
pyspark.sql.functions.curdate
pyspark.sql.functions.current_catalog
pyspark.sql.functions.current_database
pyspark.sql.functions.current_date
pyspark.sql.functions.current_schema
pyspark.sql.functions.current_time
pyspark.sql.functions.current_timestamp
pyspark.sql.functions.current_timezone
pyspark.sql.functions.current_user
pyspark.sql.functions.date_add
pyspark.sql.functions.date_diff
pyspark.sql.functions.date_format Certain esoteric formatting options are not supported, such as:
pyspark.sql.functions.date_from_unix_date
pyspark.sql.functions.date_part
pyspark.sql.functions.date_sub
pyspark.sql.functions.date_trunc
pyspark.sql.functions.dateadd
pyspark.sql.functions.datediff
pyspark.sql.functions.datepart
pyspark.sql.functions.day
pyspark.sql.functions.dayname
pyspark.sql.functions.dayofmonth
pyspark.sql.functions.dayofweek
pyspark.sql.functions.dayofyear
pyspark.sql.functions.days
pyspark.sql.functions.decode
pyspark.sql.functions.degrees
pyspark.sql.functions.dense_rank
pyspark.sql.functions.desc
pyspark.sql.functions.desc_nulls_first
pyspark.sql.functions.desc_nulls_last
pyspark.sql.functions.e
pyspark.sql.functions.element_at
pyspark.sql.functions.elt
pyspark.sql.functions.encode
pyspark.sql.functions.endswith
pyspark.sql.functions.equal_null
pyspark.sql.functions.every
pyspark.sql.functions.exists
pyspark.sql.functions.exp
pyspark.sql.functions.explode
pyspark.sql.functions.explode_outer
pyspark.sql.functions.expm1
pyspark.sql.functions.expr
pyspark.sql.functions.extract
pyspark.sql.functions.factorial
pyspark.sql.functions.filter
pyspark.sql.functions.find_in_set find_in_set only supports strings as the first argument, not dynamically another column like in pyspark.
pyspark.sql.functions.first
pyspark.sql.functions.first_value
pyspark.sql.functions.flatten
pyspark.sql.functions.floor
pyspark.sql.functions.forall
pyspark.sql.functions.format_number
pyspark.sql.functions.format_string
pyspark.sql.functions.from_csv
pyspark.sql.functions.from_json options are completely ignored
pyspark.sql.functions.from_unixtime
pyspark.sql.functions.from_utc_timestamp Currently the tz timezone argument is ignored, therefore this function is mostly useless.
pyspark.sql.functions.from_xml
pyspark.sql.functions.get
pyspark.sql.functions.get_json_object
pyspark.sql.functions.getbit
pyspark.sql.functions.greatest
pyspark.sql.functions.grouping
pyspark.sql.functions.grouping_id
pyspark.sql.functions.hash
pyspark.sql.functions.hex
pyspark.sql.functions.histogram_numeric
pyspark.sql.functions.hll_sketch_agg
pyspark.sql.functions.hll_sketch_estimate
pyspark.sql.functions.hll_union
pyspark.sql.functions.hll_union_agg
pyspark.sql.functions.hour
pyspark.sql.functions.hours
pyspark.sql.functions.hypot
pyspark.sql.functions.ifnull
pyspark.sql.functions.ilike
pyspark.sql.functions.initcap
pyspark.sql.functions.inline
pyspark.sql.functions.inline_outer
pyspark.sql.functions.input_file_block_length
pyspark.sql.functions.input_file_block_start
pyspark.sql.functions.input_file_name
pyspark.sql.functions.instr
pyspark.sql.functions.is_valid_utf8
pyspark.sql.functions.is_variant_null
pyspark.sql.functions.isnan
pyspark.sql.functions.isnotnull
pyspark.sql.functions.isnull
pyspark.sql.functions.java_method
pyspark.sql.functions.json_array_length
pyspark.sql.functions.json_object_keys
pyspark.sql.functions.json_tuple
pyspark.sql.functions.kll_sketch_agg_bigint
pyspark.sql.functions.kll_sketch_agg_double
pyspark.sql.functions.kll_sketch_agg_float
pyspark.sql.functions.kll_sketch_get_n_bigint
pyspark.sql.functions.kll_sketch_get_n_double
pyspark.sql.functions.kll_sketch_get_n_float
pyspark.sql.functions.kll_sketch_get_quantile_bigint
pyspark.sql.functions.kll_sketch_get_quantile_double
pyspark.sql.functions.kll_sketch_get_quantile_float
pyspark.sql.functions.kll_sketch_get_rank_bigint
pyspark.sql.functions.kll_sketch_get_rank_double
pyspark.sql.functions.kll_sketch_get_rank_float
pyspark.sql.functions.kll_sketch_merge_bigint
pyspark.sql.functions.kll_sketch_merge_double
pyspark.sql.functions.kll_sketch_merge_float
pyspark.sql.functions.kll_sketch_to_string_bigint
pyspark.sql.functions.kll_sketch_to_string_double
pyspark.sql.functions.kll_sketch_to_string_float
pyspark.sql.functions.kurtosis
pyspark.sql.functions.lag
pyspark.sql.functions.last
pyspark.sql.functions.last_day
pyspark.sql.functions.last_value
pyspark.sql.functions.lcase
pyspark.sql.functions.lead
pyspark.sql.functions.least
pyspark.sql.functions.left
pyspark.sql.functions.length
pyspark.sql.functions.levenshtein
pyspark.sql.functions.like
pyspark.sql.functions.listagg
pyspark.sql.functions.listagg_distinct
pyspark.sql.functions.lit
pyspark.sql.functions.ln
pyspark.sql.functions.localtimestamp
pyspark.sql.functions.locate
pyspark.sql.functions.log
pyspark.sql.functions.log10
pyspark.sql.functions.log1p
pyspark.sql.functions.log2
pyspark.sql.functions.lower
pyspark.sql.functions.lpad
pyspark.sql.functions.ltrim
pyspark.sql.functions.make_date
pyspark.sql.functions.make_dt_interval
pyspark.sql.functions.make_interval
pyspark.sql.functions.make_time
pyspark.sql.functions.make_timestamp
pyspark.sql.functions.make_timestamp_ltz
pyspark.sql.functions.make_timestamp_ntz
pyspark.sql.functions.make_valid_utf8
pyspark.sql.functions.make_ym_interval
pyspark.sql.functions.map_concat
pyspark.sql.functions.map_contains_key
pyspark.sql.functions.map_entries
pyspark.sql.functions.map_filter
pyspark.sql.functions.map_from_arrays
pyspark.sql.functions.map_from_entries
pyspark.sql.functions.map_keys
pyspark.sql.functions.map_values
pyspark.sql.functions.map_zip_with
pyspark.sql.functions.mask
pyspark.sql.functions.max
pyspark.sql.functions.max_by
pyspark.sql.functions.md5
pyspark.sql.functions.mean
pyspark.sql.functions.median
pyspark.sql.functions.min
pyspark.sql.functions.min_by
pyspark.sql.functions.minute
pyspark.sql.functions.mode
pyspark.sql.functions.monotonically_increasing_id
pyspark.sql.functions.month
pyspark.sql.functions.monthname
pyspark.sql.functions.months
pyspark.sql.functions.months_between The parameter roundOff is not honored.
pyspark.sql.functions.named_struct
pyspark.sql.functions.nanvl
pyspark.sql.functions.negate
pyspark.sql.functions.negative
pyspark.sql.functions.next_day
pyspark.sql.functions.now
pyspark.sql.functions.nth_value
pyspark.sql.functions.ntile
pyspark.sql.functions.nullif
pyspark.sql.functions.nullifzero
pyspark.sql.functions.nvl
pyspark.sql.functions.nvl2
pyspark.sql.functions.octet_length
pyspark.sql.functions.overlay
pyspark.sql.functions.parse_json
pyspark.sql.functions.parse_url
pyspark.sql.functions.percent_rank
pyspark.sql.functions.percentile The frequency argument is not honored.
pyspark.sql.functions.percentile_approx The accuracy argument is not honored.
pyspark.sql.functions.pi
pyspark.sql.functions.pmod
pyspark.sql.functions.posexplode
pyspark.sql.functions.posexplode_outer
pyspark.sql.functions.position
pyspark.sql.functions.positive
pyspark.sql.functions.pow
pyspark.sql.functions.power
pyspark.sql.functions.printf
pyspark.sql.functions.product
pyspark.sql.functions.quarter
pyspark.sql.functions.quote
pyspark.sql.functions.radians
pyspark.sql.functions.raise_error
pyspark.sql.functions.rand The seed value is accepted for API compatibility, but is unused. Even if set, the function will not be reproducible.
pyspark.sql.functions.randn The seed value is accepted for API compatibility, but is unused. Even if set, the function will not be reproducible.
pyspark.sql.functions.random
pyspark.sql.functions.randstr The seed argument is not honored. Output is lowercase-only.
pyspark.sql.functions.rank
pyspark.sql.functions.reduce
pyspark.sql.functions.reflect
pyspark.sql.functions.regexp
pyspark.sql.functions.regexp_count
pyspark.sql.functions.regexp_extract
pyspark.sql.functions.regexp_extract_all For technical reasons, the first argument is called str_ instead of str.
pyspark.sql.functions.regexp_instr
pyspark.sql.functions.regexp_like
pyspark.sql.functions.regexp_replace
pyspark.sql.functions.regexp_substr
pyspark.sql.functions.regr_avgx
pyspark.sql.functions.regr_avgy
pyspark.sql.functions.regr_count
pyspark.sql.functions.regr_intercept
pyspark.sql.functions.regr_r2
pyspark.sql.functions.regr_slope
pyspark.sql.functions.regr_sxx
pyspark.sql.functions.regr_sxy
pyspark.sql.functions.regr_syy
pyspark.sql.functions.repeat
pyspark.sql.functions.replace
pyspark.sql.functions.reverse
pyspark.sql.functions.right
pyspark.sql.functions.rint
pyspark.sql.functions.rlike
pyspark.sql.functions.round
pyspark.sql.functions.row_number
pyspark.sql.functions.rpad
pyspark.sql.functions.rtrim
pyspark.sql.functions.schema_of_csv
pyspark.sql.functions.schema_of_json
pyspark.sql.functions.schema_of_variant
pyspark.sql.functions.schema_of_variant_agg
pyspark.sql.functions.schema_of_xml
pyspark.sql.functions.sec
pyspark.sql.functions.second
pyspark.sql.functions.sentences
pyspark.sql.functions.sequence
pyspark.sql.functions.session_user
pyspark.sql.functions.session_window
pyspark.sql.functions.sha
pyspark.sql.functions.sha1
pyspark.sql.functions.sha2 Only numBits 256 or 512 are supported.
pyspark.sql.functions.shiftLeft
pyspark.sql.functions.shiftRight
pyspark.sql.functions.shiftRightUnsigned
pyspark.sql.functions.shiftleft
pyspark.sql.functions.shiftright
pyspark.sql.functions.shiftrightunsigned
pyspark.sql.functions.shuffle
pyspark.sql.functions.sign
pyspark.sql.functions.signum
pyspark.sql.functions.sin
pyspark.sql.functions.sinh
pyspark.sql.functions.size
pyspark.sql.functions.skewness
pyspark.sql.functions.slice
pyspark.sql.functions.some
pyspark.sql.functions.sort_array Descending sort (asc=False) is not supported. Arrays are always sorted in ascending order.
pyspark.sql.functions.soundex
pyspark.sql.functions.spark_partition_id
pyspark.sql.functions.split The limit argument is not honored.
pyspark.sql.functions.split_part
pyspark.sql.functions.sqrt
pyspark.sql.functions.st_asbinary
pyspark.sql.functions.st_geogfromwkb
pyspark.sql.functions.st_geomfromwkb
pyspark.sql.functions.st_setsrid
pyspark.sql.functions.st_srid
pyspark.sql.functions.stack
pyspark.sql.functions.startswith
pyspark.sql.functions.std
pyspark.sql.functions.stddev
pyspark.sql.functions.stddev_pop
pyspark.sql.functions.stddev_samp
pyspark.sql.functions.str_to_map
pyspark.sql.functions.string_agg
pyspark.sql.functions.string_agg_distinct
pyspark.sql.functions.struct
pyspark.sql.functions.substr
pyspark.sql.functions.substring
pyspark.sql.functions.substring_index Negative counts are not supported.
pyspark.sql.functions.sum
pyspark.sql.functions.sumDistinct
pyspark.sql.functions.sum_distinct
pyspark.sql.functions.tan
pyspark.sql.functions.tanh
pyspark.sql.functions.theta_difference
pyspark.sql.functions.theta_intersection
pyspark.sql.functions.theta_intersection_agg
pyspark.sql.functions.theta_sketch_agg
pyspark.sql.functions.theta_sketch_estimate
pyspark.sql.functions.theta_union
pyspark.sql.functions.theta_union_agg
pyspark.sql.functions.time_diff
pyspark.sql.functions.time_trunc
pyspark.sql.functions.timestamp_add
pyspark.sql.functions.timestamp_diff
pyspark.sql.functions.timestamp_micros
pyspark.sql.functions.timestamp_millis
pyspark.sql.functions.timestamp_seconds
pyspark.sql.functions.toDegrees
pyspark.sql.functions.toRadians
pyspark.sql.functions.to_binary
pyspark.sql.functions.to_char
pyspark.sql.functions.to_csv
pyspark.sql.functions.to_date
pyspark.sql.functions.to_json
pyspark.sql.functions.to_number
pyspark.sql.functions.to_time
pyspark.sql.functions.to_timestamp
pyspark.sql.functions.to_timestamp_ltz Using a non-lieral column is not supported for the format string.
pyspark.sql.functions.to_timestamp_ntz Using a non-lieral column is not supported for the format string.
pyspark.sql.functions.to_unix_timestamp
pyspark.sql.functions.to_utc_timestamp
pyspark.sql.functions.to_varchar
pyspark.sql.functions.to_variant_object
pyspark.sql.functions.to_xml
pyspark.sql.functions.transform
pyspark.sql.functions.transform_keys
pyspark.sql.functions.transform_values
pyspark.sql.functions.translate
pyspark.sql.functions.trim
pyspark.sql.functions.trunc
pyspark.sql.functions.try_add
pyspark.sql.functions.try_aes_decrypt
pyspark.sql.functions.try_avg
pyspark.sql.functions.try_divide
pyspark.sql.functions.try_element_at
pyspark.sql.functions.try_make_interval
pyspark.sql.functions.try_make_timestamp
pyspark.sql.functions.try_make_timestamp_ltz
pyspark.sql.functions.try_make_timestamp_ntz
pyspark.sql.functions.try_mod
pyspark.sql.functions.try_multiply
pyspark.sql.functions.try_parse_json
pyspark.sql.functions.try_parse_url
pyspark.sql.functions.try_reflect
pyspark.sql.functions.try_subtract
pyspark.sql.functions.try_sum
pyspark.sql.functions.try_to_binary
pyspark.sql.functions.try_to_date
pyspark.sql.functions.try_to_number
pyspark.sql.functions.try_to_time
pyspark.sql.functions.try_to_timestamp
pyspark.sql.functions.try_url_decode
pyspark.sql.functions.try_validate_utf8
pyspark.sql.functions.try_variant_get
pyspark.sql.functions.typeof
pyspark.sql.functions.ucase
pyspark.sql.functions.udf
pyspark.sql.functions.udtf
pyspark.sql.functions.unbase64
pyspark.sql.functions.unhex
pyspark.sql.functions.uniform
pyspark.sql.functions.unix_date
pyspark.sql.functions.unix_micros
pyspark.sql.functions.unix_millis
pyspark.sql.functions.unix_seconds
pyspark.sql.functions.unix_timestamp
pyspark.sql.functions.unwrap_udt
pyspark.sql.functions.upper
pyspark.sql.functions.url_decode
pyspark.sql.functions.url_encode
pyspark.sql.functions.user
pyspark.sql.functions.uuid
pyspark.sql.functions.validate_utf8
pyspark.sql.functions.var_pop
pyspark.sql.functions.var_samp
pyspark.sql.functions.variance
pyspark.sql.functions.variant_get
pyspark.sql.functions.version
pyspark.sql.functions.weekday
pyspark.sql.functions.weekofyear
pyspark.sql.functions.when
pyspark.sql.functions.width_bucket
pyspark.sql.functions.window
pyspark.sql.functions.window_time
pyspark.sql.functions.xpath
pyspark.sql.functions.xpath_boolean
pyspark.sql.functions.xpath_double
pyspark.sql.functions.xpath_float
pyspark.sql.functions.xpath_int
pyspark.sql.functions.xpath_long
pyspark.sql.functions.xpath_number
pyspark.sql.functions.xpath_short
pyspark.sql.functions.xpath_string
pyspark.sql.functions.xxhash64
pyspark.sql.functions.year
pyspark.sql.functions.years
pyspark.sql.functions.zeroifnull
pyspark.sql.functions.zip_with

DataTypes (28/37 = 76%)

API Implemented Notes
pyspark.sql.types.AnsiIntervalType
pyspark.sql.types.AnyTimeType
pyspark.sql.types.ArrayType
pyspark.sql.types.AtomicType
pyspark.sql.types.BinaryType
pyspark.sql.types.BooleanType
pyspark.sql.types.ByteType
pyspark.sql.types.CalendarIntervalType
pyspark.sql.types.CharType
pyspark.sql.types.DataType
pyspark.sql.types.DateType
pyspark.sql.types.DatetimeType
pyspark.sql.types.DayTimeIntervalType
pyspark.sql.types.DecimalType
pyspark.sql.types.DoubleType
pyspark.sql.types.FloatType
pyspark.sql.types.FractionalType
pyspark.sql.types.GeographyType
pyspark.sql.types.GeometryType
pyspark.sql.types.IntegerType
pyspark.sql.types.IntegralType
pyspark.sql.types.LongType
pyspark.sql.types.MapType
pyspark.sql.types.NullType
pyspark.sql.types.NumericType
pyspark.sql.types.ShortType
pyspark.sql.types.SpatialType
pyspark.sql.types.StringType
pyspark.sql.types.StructField
pyspark.sql.types.StructType
pyspark.sql.types.TimeType
pyspark.sql.types.TimestampNTZType
pyspark.sql.types.TimestampType
pyspark.sql.types.UserDefinedType
pyspark.sql.types.VarcharType
pyspark.sql.types.VariantType
pyspark.sql.types.YearMonthIntervalType