This article explains why GPU acceleration isn't currently available for in Nuke running on macOS 10.13 High Sierra with an NVIDIA card.
In order to use Nuke's GPU acceleration with an NVIDIA graphics card on macOS you need to have a compatible graphics card, up to date GPU drivers and additional CUDA drivers installed.
NVIDIA have released general graphics drivers for macOS High Sierra 10.13 but they have not yet released CUDA drivers which are compatible with macOS 10.13 High Sierra. This means that it isn't currently possible to use GPU acceleration in Nuke on macOS 10.13 if you have an NVIDIA card until compatible CUDA drivers are released.
We will update this article once new CUDA drivers are available and we have more information.
NOTE: CUDA drivers are available for macOS 10.12 Sierra therefore GPU Acceleration is possible with Nuke if you are running macOS 10.12.
The full system requirements for GPU acceleration for Nuke on MacOS can be found in the Installation & Licensing - Mac OS X and macOS - Requirements for GPU Acceleration section of the Nuke User Guide.