Just for some more information I gave the current tip of the amd-staging-4.11 branch on https://cgit.freedesktop.org/~agd5f/linux/ (commit 308514e6861e) a try and it was still having the same problems driving my monitor at 5120x2880.