Convert Python datetime object to string

There are multiple ways to display date and time values with Python, however not all of them are easy to read.

For example, when you collect a timestamp column from a DataFrame and save it as a Python variable, the value is stored as a datetime object. If you are not familiar with the datetime object format, it is not as easy to read as the common YYYY-MM-DD HH:MM:SS format.

If you wanted to print the date and time, or maybe use it for timestamp validation, you can convert the datetime object to a string. This automatically converts the datetime object into a common time format.

In this article, we show you how to display the timestamp as a column value, before converting it to a datetime object, and finally, a string value.

Display timestamp as a column value

To display the current timestamp as a column value, you should call current_timestamp().

This provides the date and time as of the moment it is called.

from pyspark.sql.functions import *

Sample output:

Display current_timestamp as a column value

Assign timestamp to datetime object

Instead of displaying the date and time in a column, you can assign it to a variable.

mydate = spark.range(1).withColumn("date",current_timestamp()).select("date").collect()[0][0]

Once this assignment is made, you can call the variable to display the stored date and time value as a datetime object.


Sample output:

datetime.datetime(2021, 6, 25, 11, 0, 56, 813000)


The date and time is current as of the moment it is assigned to the variable as a datetime object, but the datetime object value is static unless a new value is assigned.

Convert to string

You can convert the datetime object to a string by calling str() on the variable. Calling str() just converts the datetime object to a string. It does not update the value with the current date and time.


Sample output:

'2021-06-25 11:00:56.813000'