pyspark.sql.functions.array_contains#

pyspark.sql.functions.array_contains(col, value)[source]#

Collection function: Returns true if the array contains the value, false if not. Returns null if the array or value is null, or if the value is not found and the array contains a null element.

Added in version 1.5.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

colColumn or str: The target column containing the arrays. A column that evaluates to an array.
value: The value or column to check for in the array. A column of the same type as the array elements.

Returns:

Column: A new Column of Boolean type, where each value indicates whether the corresponding array from the input column contains the specified value. Returns a column that evaluates to a boolean.

See also

pyspark.sql.functions.array_position()

Examples

Example 1: Basic usage of array_contains function.

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"],), ([],)], ['data'])
>>> df.select(sf.array_contains(df.data, "a")).show()
+-----------------------+
|array_contains(data, a)|
+-----------------------+
|                   true|
|                  false|
+-----------------------+

Example 2: Usage of array_contains function with a column.

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], "c"),
...                            (["c", "d", "e"], "d"),
...                            (["e", "a", "c"], "b")], ["data", "item"])
>>> df.select(sf.array_contains(df.data, sf.col("item"))).show()
+--------------------------+
|array_contains(data, item)|
+--------------------------+
|                      true|
|                      true|
|                     false|
+--------------------------+

Example 3: Attempt to use array_contains function with a null array.

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(None,), (["a", "b", "c"],)], ['data'])
>>> df.select(sf.array_contains(df.data, "a")).show()
+-----------------------+
|array_contains(data, a)|
+-----------------------+
|                   NULL|
|                   true|
+-----------------------+

Example 4: Usage of array_contains with an array column containing null values.

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", None, "c"],)], ['data'])
>>> df.select(sf.array_contains(df.data, "a")).show()
+-----------------------+
|array_contains(data, a)|
+-----------------------+
|                   true|
+-----------------------+

Example 5: Value absent from an array that contains a null element returns NULL.

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", None, "c"],)], ['data'])
>>> df.select(sf.array_contains(df.data, "b")).show()
+-----------------------+
|array_contains(data, b)|
+-----------------------+
|                   NULL|
+-----------------------+