There are multiple ways to display date and time values with Python, however not all of them are easy to read.
For example, when you collect a timestamp column from a DataFrame and save it as a Python variable, the value is stored as a datetime object. If you are not familiar with the datetime object format, it is not as easy to read as the common YYYY-MM-DD HH:MM:SS format.
If you wanted to print the date and time, or maybe use it for timestamp validation, you can convert the datetime object to a string. This automatically converts the datetime object into a common time format.
In this article, we show you how to display the timestamp as a column value, before converting it to a datetime object, and finally, a string value.
Display timestamp as a column value
To display the current timestamp as a column value, you should call current_timestamp().
This provides the date and time as of the moment it is called.
%python from pyspark.sql.functions import * display(spark.range(1).withColumn("date",current_timestamp()).select("date"))
Assign timestamp to datetime object
Instead of displaying the date and time in a column, you can assign it to a variable.
%python mydate = spark.range(1).withColumn("date",current_timestamp()).select("date").collect()
Once this assignment is made, you can call the variable to display the stored date and time value as a datetime object.
datetime.datetime(2021, 6, 25, 11, 0, 56, 813000)
Convert to string
You can convert the datetime object to a string by calling str() on the variable. Calling str() just converts the datetime object to a string. It does not update the value with the current date and time.