Looking at libpciaccess, the configure.ac seems to require xorg-macros when it
is not needed in any way shape or form.  Removing the xorg-macros still results
in a functioning build.

I did have to disable the man pages, this may not be related though:

make[2]: Entering directory `/home/e1529750/Code/libpciaccess-0.13.2/man'
/bin/sed  < scanpci.man > scanpci.
Usage: /bin/sed [OPTION]... {script-only-if-no-other-script} [input-file]...

  -n, --quiet, --silent
                 suppress automatic printing of pattern space


Mentioning this as libdrm is supposed to not rely on xorg to function, but it
relies on libpciaccess which requires xorg-macros.