scott@bjorn3d SSC Member Total Posts : 607

Reward points : 0

Joined:

Status: offline

Ribbons : 1 permalink)





"Hey Scott,



The team has been working on this around the clock to unblock the folding community and we believe the root cause is a bug in the Folding@home dependent OpenMM software. We’ve engaged with them to see if they agree and get changes done by 12/14.



In case we can’t get those changes done in the app soon, we’re working on a fall-back option in our JIT compiler, based on an app profile to disable an optimization that is exposing the synchronization bug in the app. We’re targeting to get that driver also posted publically on 12/14.



I’ll let you know if either of the above doesn’t happen for some reason and the new ETA.



Thanks for your patience!



Regards,

Team NV"



12/14 Update



Hey Scott,



Our efforts to get a resolution out by 12/14 has run into some roadblocks. The change suggestion we made for OpenMM needs more tweaks and our driver side workaround is close to being done but not complete yet. We plan to post a hotfix driver to unblock the Folding@Home users as soon as we have a workaround to share (within the next few days).



We’ll keep you posted.



Regards,

Team NV

post edited by scott@bjorn3d - Got this from a connection at NVIDIA, guess the old days of owning Bjorn3d.com helps out a little."Hey Scott,The team has been working on this around the clock to unblock the folding community and we believe the root cause is a bug in the Folding@home dependent OpenMM software. We’ve engaged with them to see if they agree and get changes done by 12/14.In case we can’t get those changes done in the app soon, we’re working on a fall-back option in our JIT compiler, based on an app profile to disable an optimization that is exposing the synchronization bug in the app. We’re targeting to get that driver also posted publically on 12/14.I’ll let you know if either of the above doesn’t happen for some reason and the new ETA.Thanks for your patience!Regards,Team NV"12/14 UpdateHey Scott,Our efforts to get a resolution out by 12/14 has run into some roadblocks. The change suggestion we made for OpenMM needs more tweaks and our driver side workaround is close to being done but not complete yet. We plan to post a hotfix driver to unblock the Folding@Home users as soon as we have a workaround to share (within the next few days).We’ll keep you posted.Regards,Team NV



#1 25 Replies Related Threads

Mekhed iCX Member Total Posts : 410

Reward points : 0

Joined:

Location: Rocky Mountain High

Status: offline

Ribbons : 1

Re: Got word from NVIDIA and this issue they are working on. permalink) That sounds encouraging.





"Kaos" ASUS RAMPAGE V EXTREME | Intel Core i7-5930K | Corsair H110i GTX | 2x 1080TI Hybrid & 1080 Hybrid | HyperX 32GB 3000 | Supernova 1200 | Corsair 780T

"BigDawg" Asus Maximus VI Formula | i7-4790k | Corsair H100i | 2x RTX 2080 Ti FTW3 Ultra | Mushkin Redline 16GB 2400 | Supernova 1300 | Antec 1200

#2

Chris21010 FTW Member Total Posts : 1659

Reward points : 0

Joined:

Status: offline

Ribbons : 9

Re: Got word from NVIDIA and this issue they are working on. permalink) awesome!





#3

DutchForce CLASSIFIED Member Total Posts : 2359

Reward points : 0

Joined:

Location: Netherlands

Status: offline

Ribbons : 52 Re: Got word from NVIDIA and this issue they are working on. permalink)

Thanks for your effort to get more (good) info from NVidia, Scott, it's very much appreciated !!!





Folding Rig: Lian-Li PC-P50 | Corsair HX850 | MSI 890FXA-GD70 | AMD X4-965 BE | GTX980 Ti SC & GTX1070 SC

#4

drougnor CLASSIFIED Member Total Posts : 2632

Reward points : 0

Joined:

Location: FL USA /Affiliate MCK692UUS /Assoc HPW7E9T81P8KCCH

Status: offline

Ribbons : 18

Re: Got word from NVIDIA and this issue they are working on. permalink)



**Edit - With a little coffee and thought, I realize they're saying it's NVIDIA's code optimization that forced the bug to come out of hiding, so disabling it means they'll go back to the code that is currently working, therefore NOT giving us worse numbers. But, if that optimization is allowed to go through properly, we could see BETTER numbers. So, let's hope OpenMM fixes it!**



I really do hope that they'll be able to get things straightened out properly, obviously, but am ready to keep the current driver installer on hand just in case.



d

post edited by drougnor - Whenever I see a line like "we’re working on a fall-back option in our JIT compiler, based on an app profile to disable an optimization that is exposing the synchronization bug in the app" I have to wonder what this means for our production numbers?**Edit - With a little coffee and thought, I realize they're saying it's NVIDIA's code optimization that forced the bug to come out of hiding, so disabling it means they'll go back to the code that is currently working, therefore NOT giving us worse numbers. But, if that optimization is allowed to go through properly, we could see BETTER numbers. So, let's hope OpenMM fixes it!**I really do hope that they'll be able to get things straightened out properly, obviously, but am ready to keep the current driver installer on hand just in case.



#5

HK-Steve CLASSIFIED Member Total Posts : 2253

Reward points : 0

Joined:

Location: Switzerland

Status: offline

Ribbons : 2 Re: Got word from NVIDIA and this issue they are working on. permalink) fingerscrossed









#6

scott@bjorn3d SSC Member Total Posts : 607

Reward points : 0

Joined:

Status: offline

Ribbons : 1 Re: Got word from NVIDIA and this issue they are working on. permalink) Oh yeah I keep copies of all working drivers.







#7

rjohnson11 EVGA Forum Moderator Total Posts : 78784

Reward points : 0

Joined:

Location: Europe

Status: online

Ribbons : 71

Re: Got word from NVIDIA and this issue they are working on. permalink) Yes this is the same info I've been given from NVIDIA so hopefully this can be resolved soon.



Specs: AMD Ryzen 9 3950X, Corsair MP600 M.2 , 64GB Corsair RGB Dominator, ASUS Crosshair VIII Hero, Corsair 1000D, EVGA RTX 2070 FTW 3



Associate Code: H5U80QBH6BH0AXF







#8

bcavnaugh The Crunchinator Total Posts : 38112

Reward points : 0

Joined:

Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H

Status: offline

Ribbons : 276

Re: Got word from NVIDIA and this issue they are working on. permalink)

https://folding.stanford.edu/home/core-21-openmm-gpu-issues-with-new-windows-nvidia-drivers-375-57-and-375-63/

Still Points to NVIDIA and Not to (OpenMM) or Their Coding.

post edited by bcavnaugh - I will believe it when I see it.Still Points to NVIDIA and Not to (OpenMM) or Their Coding.

Affiliate Code:







Affiliate Code: E5L3CTGE12 & Associate Code: 9E88QK5L7811G3H #9

Chris21010 FTW Member Total Posts : 1659

Reward points : 0

Joined:

Status: offline

Ribbons : 9

Re: Got word from NVIDIA and this issue they are working on. permalink) that update is 2 weeks old and may not still be relevant. either way as long as we can get something working with relatively the same performance and allow people to play the newer titles will be a win/win no mater who is at fault.





#10

bcavnaugh The Crunchinator Total Posts : 38112

Reward points : 0

Joined:

Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H

Status: offline

Ribbons : 276

Re: Got word from NVIDIA and this issue they are working on. permalink) Chris21010

that update is 2 weeks old and may not still be relevant. either way as long as we can get something working with relatively the same performance and allow people to play the newer titles will be a win/win no mater who is at fault.





Yes two Weeks and Still No New info from Folding @ Home

post edited by bcavnaugh - Yes two Weeks and Still No New info from Folding @ Home

Affiliate Code:







Affiliate Code: E5L3CTGE12 & Associate Code: 9E88QK5L7811G3H #11

kougar CLASSIFIED Member Total Posts : 2019

Reward points : 0

Joined:

Location: Texas

Status: offline

Ribbons : 9

Re: Got word from NVIDIA and this issue they are working on. permalink) Thanks for the update!! Props to NVIDIA for debugging the OpenMM software which is out of their purview. Very glad to see some concrete dates





Have water, will cool.



Don't forget to use an Associate Code at checkout for extra savings! Associate Code: FWBXGF17CKOMCIV

#12

ipkha CLASSIFIED Member Total Posts : 2234

Reward points : 0

Joined:

Location: Indiana

Status: offline

Ribbons : 6

Re: Got word from NVIDIA and this issue they are working on. permalink) Stanford has never once accepted fault in their coding. I've seen work units fail seconds in, and their software says I must have been overclocking. Never-ending that 90% of the other tasks work just fine. Frankly, this is an open source problem in general. No one wants to admit that their code may broken. I'm just glad there is a fix available from Nvidia, it'll be months before Stanford fixes OpenMM. And even when they do fix it, the folding folks would still have to update the core to use a newer version. It's a big mess all around.



#13

drougnor CLASSIFIED Member Total Posts : 2632

Reward points : 0

Joined:

Location: FL USA /Affiliate MCK692UUS /Assoc HPW7E9T81P8KCCH

Status: offline

Ribbons : 18

Re: Got word from NVIDIA and this issue they are working on. permalink) Tomorrow's the day, either direction. Cross those fingers, folks!



d





#14

bcavnaugh The Crunchinator Total Posts : 38112

Reward points : 0

Joined:

Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H

Status: offline

Ribbons : 276

Re: Got word from NVIDIA and this issue they are working on. permalink) drougnor

Tomorrow's the day, either direction. Cross those fingers, folks!



d





Core 22? or New Driver?

post edited by bcavnaugh - Core 22? or New Driver?

Affiliate Code:







Affiliate Code: E5L3CTGE12 & Associate Code: 9E88QK5L7811G3H #15

Chris21010 FTW Member Total Posts : 1659

Reward points : 0

Joined:

Status: offline

Ribbons : 9

Re: Got word from NVIDIA and this issue they are working on. permalink) well new driver for sure, but i also hope that Stanford fixes this bug in core 21 sooner than never.





#16

drougnor CLASSIFIED Member Total Posts : 2632

Reward points : 0

Joined:

Location: FL USA /Affiliate MCK692UUS /Assoc HPW7E9T81P8KCCH

Status: offline

Ribbons : 18

Re: Got word from NVIDIA and this issue they are working on. permalink) With luck it'll be a 'simple' (depending on where in the code the bug is and what is needed to straighten it out) update to the core that will auto update without us even knowing. That's the kind of fix PG tends to prefer - No fuss, no muss.



d





#17

bcavnaugh The Crunchinator Total Posts : 38112

Reward points : 0

Joined:

Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H

Status: offline

Ribbons : 276

Re: Got word from NVIDIA and this issue they are working on. permalink) That is good to hear, I will stage my GTX 980 and GTX 1080 Rigs with the Current Driver and Stand By.



Affiliate Code:







Affiliate Code: E5L3CTGE12 & Associate Code: 9E88QK5L7811G3H #18

EllBrad New Member Total Posts : 69

Reward points : 0

Joined:

Location: Canada

Status: offline

Ribbons : 0 Re: Got word from NVIDIA and this issue they are working on.--UPDATED 12/14 permalink) scott@bjorn3d Hey Scott,



Our efforts to get a resolution out by 12/14 has run into some roadblocks. The change suggestion we made for OpenMM needs more tweaks and our driver side workaround is close to being done but not complete yet. We plan to post a hotfix driver to unblock the Folding@Home users as soon as we have a workaround to share (within the next few days).



We’ll keep you posted.



Regards,

Team NV













i5 4690k @ 4.5ghz | EVGA GTX 1070SC @ 2.1ghz | 16gb DDR3 @ 1866mhz | Acer XB270HU

#19

drougnor CLASSIFIED Member Total Posts : 2632

Reward points : 0

Joined:

Location: FL USA /Affiliate MCK692UUS /Assoc HPW7E9T81P8KCCH

Status: offline

Ribbons : 18

Re: Got word from NVIDIA and this issue they are working on.--UPDATED 12/14 permalink) Updates are good. Not as good as a resolution, but it IS an update!



d





#20

scott@bjorn3d SSC Member Total Posts : 607

Reward points : 0

Joined:

Status: offline

Ribbons : 1 Re: Got word from NVIDIA and this issue they are working on.--UPDATED 12/14 permalink) The good news is the CEO of NVIDIA is being CC on the progress.





#21

bcavnaugh The Crunchinator Total Posts : 38112

Reward points : 0

Joined:

Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H

Status: offline

Ribbons : 276

Re: Got word from NVIDIA and this issue they are working on.--UPDATED 12/14 permalink)

Workarounds are only Band-Aids and Fall Off they are Not Fixes.



EllBrad

scott@bjorn3d

Hey Scott,



Our efforts to get a resolution out by 12/14 has run into some roadblocks. The change suggestion we made for OpenMM needs more tweaks and our driver side workaround is close to being done but not complete yet. We plan to post a hotfix driver to unblock the Folding@Home users as soon as we have a workaround to share (within the next few days).



We’ll keep you posted.



Regards,

Team NV







Did



Misc Notes Re: Folding@Home Issue It was posted yesterday that this driver should've fixed the Folding@Home issue.

Seems like they hit a "roadblock" and the fix did not make it in time to this driver. A hotfix will be deployed instead.

Here's the update from the post:

12/14 Update

Hey Scott,

Our efforts to get a resolution out by 12/14 has run into some roadblocks. The change suggestion we made for OpenMM needs more tweaks and our driver side workaround is close to being done but not complete yet. We plan to post a hotfix driver to unblock the Folding@Home users as soon as we have a workaround to share (within the next few days).

We’ll keep you posted.

Regards, Team NV



Funny that the link above Link links back here



http://us.download.nvidia.com/Windows/376.33/376.33-desktop-win10-64bit-international-whql.exe

http://us.download.nvidia.com/Windows/376.33/376.33-desktop-win8-win7-64bit-international-whql.exe



post edited by bcavnaugh - Not happy with this "our driver side workaround is close to being done but not complete yet"Workarounds are only Band-Aids and Fall Off they are Not Fixes.Did 376.33 Fix it As seen below.It was posted yesterday that this driver should've fixed the Folding@Home issue. Link here Seems like they hit a "roadblock" and the fix did not make it in time to this driver. A hotfix will be deployed instead.Here's the update from the post:Funny that the link above Link links back here http://forums.evga.com/Got-word-from-NVIDIA-and-this-issue-they-are-working-onUPDATED-1214-m2593454.aspx#2593454 and not to https://bjorn3d.com/

Affiliate Code:







Affiliate Code: E5L3CTGE12 & Associate Code: 9E88QK5L7811G3H #22

Ali Man iCX Member Total Posts : 258

Reward points : 0

Joined:

Status: offline

Ribbons : 0 Re: Got word from NVIDIA and this issue they are working on.--UPDATED 12/14 permalink) Anyone tried the latest driver?

#23

bcavnaugh The Crunchinator Total Posts : 38112

Reward points : 0

Joined:

Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H

Status: offline

Ribbons : 276

Re: Got word from NVIDIA and this issue they are working on.--UPDATED 12/14 permalink) Ali Man

Anyone tried the latest driver?





They did not fix the Folding Issue as stated in the Post above, still waiting for the Hotfix File to be released.

post edited by bcavnaugh - They did not fix the Folding Issue as stated in the Post above, still waiting for the Hotfix File to be released.

Affiliate Code:







Affiliate Code: E5L3CTGE12 & Associate Code: 9E88QK5L7811G3H #24