GPU dropping off once mining start

[lockon]

Below is my rig setup
Motherboard : Z87-G45 GAMING (MS-7821) MSI (V1.9 07/21/2014)
CPU : 4 × Intel(R) Core™ i5-4690 CPU @ 3.50GHz AES
Disk Model : ATA Micron C400 Real 32.0GB
RAM : 8GB
GPU : GTX 1070 x 6
OS : 0.6-224@230909
Driver : 525.125.06

Run the following command:
grep -a Xid /var/log/syslog | tail -n 10

and got the following feedback below
kernel: [ 34.648324][ T3487] NVRM: Xid (PCI:0000:08:00): 79, pid=3487, GPU has fallen off the bus.

On Rig boot up, GPU is able detect with power and temp but once the mining start it will drop off as pre the error message above.

I have done the following in order:
1)Disable all overclock and power setting
2)switch to anther miner application
3)Replace the whole set of PCIE 1x Raiser
4)Swapped the mention issue GPU to another working PCIE slot and the other working GPU to the mention issue PCIE slot
5)replaced the storage
6)Upgrade HiveOS version
7)Update GPU driver

any advise is welcome

[keaton_hiveon]

Does the gpu work directly on the motherboard?

[lockon]

I have not try it.
the setup is currently in watercooled open loop so it will be abit problematic to do it.

but i did plug it ( the gpu and the raiser set) to a known working slot and the known working GPU (w/ the whole raiser set) to the mention issue slot. and the result was the same.
Once the mining start, the working gpu on the mention issue slot will drop off

[keaton_hiveon]

might need to get soft lines to try it then, thats where i would start though. if it still errors directly on the board and youve tried multiple psus id imagine the card is at fault

[lockon]

what do you mean by “multiple psus id imagine the card is at fault” ?

[keaton_hiveon]

Rule out each variable to determine the issue, psu, risers, slots, card itself etc

[Farmingfarmer]

What’s your power supply? Could be a power issue. Try powerlimitng the cards as low as you can.