I use SOURCE_DATE_EPOCH=0 FORCE_SOURCE_DATE=1 xelatex test
to create reproducible PDF files.
Since I want to understand some detail I wrote test.tex with uncompression.
%\DocumentMetadata{uncompress}
\documentclass[a4paper]{article}
\special{dvipdfmx:config z 0}
\begin{document}
test
\end{document}
Of course, I do not like xelatex specific command \special{dvipdfmx:config z 0} to refrain from compression.
All works fine, a diff shows, results are the same when compiling twice.
If I replace the \special command by the more general \DocumentMetadata,
then a diff unveils
$ diff -a test.pdf testX.pdf
220c220
< <rdf:li>2023-12-06T21:36:39Z</rdf:li>
---
> <rdf:li>2023-12-06T21:36:32Z</rdf:li>
226,228c226,228
< <xmp:CreateDate>2023-12-06T21:36:39Z</xmp:CreateDate>
< <xmp:ModifyDate>2023-12-06T21:36:39Z</xmp:ModifyDate>
< <xmp:MetadataDate>2023-12-06T21:36:39Z</xmp:MetadataDate>
---
> <xmp:CreateDate>2023-12-06T21:36:32Z</xmp:CreateDate>
> <xmp:ModifyDate>2023-12-06T21:36:32Z</xmp:ModifyDate>
> <xmp:MetadataDate>2023-12-06T21:36:32Z</xmp:MetadataDate>
230c230
< <xmpMM:InstanceID>uuid:70381c59-1666-4bc2-80a7-7e613d06022e</xmpMM:InstanceID>
---
> <xmpMM:InstanceID>uuid:2c37e4bc-7448-455d-8793-19c3e55d01ca</xmpMM:InstanceID>
256c256
< <</Creator(LaTeX)/Producer(xetex-0.999995)/CreationDate(D:20231206213639Z)/ModDate(D:20231206213639Z)>>
---
> <</Creator(LaTeX)/Producer(xetex-0.999995)/CreationDate(D:20231206213632Z)/ModDate(D:20231206213632Z)>>
exposing the correct CreationDate includint eh time also
and there are more timestamps probably introduced by \DocumentMetadata.
Comparing one version with \special and one with \DocumentMetadata is also instructive.
It is a bit long but the \special version shows:
< <</Creator( XeTeX output 1970.01.01:0000)/Producer(xdvipdfmx \(20220710\))/CreationDate(D:19700101000000-00'00')>>
and CreationDate(D:19700101000000-00'00') corresponds with SOURCE_DATE_EPOCH=0, correct.
By the way, replacing \DocumentMetadata{uncompress} by \DocumentMetadata{pdfversion=1.8} corrupts reproducibility also.
Also interesting: This behavior is not specific to xelatex,
also lualatex and pdflatex behave essentially like that.