14.3.1                    Subregions

In QW-Simulator MultiGPU, the user establishes how many devices will be used for particular project calculation. The decision is made beforehand. Every device is running its own part of the project called Subregion. User can create them by adding in editor of choice a plane called Subregion Border. Each Subregion contains an appropriate region between project boundaries and Subregion Borders plus a small margin – 3 cells thick – overlapping the neighbouring Subregions, used for synchronisation between them. Project exported with Subregions can be simulated only in QW-Simulator MultiGPU. To run simulation using non QW-Simulator MultiGPU the partitioning should be turned off in Export Options dialogue and project needs to be re-exported.

Subregions are given their names according to their relative position in the project. The key is "sreg_xNyMzK" where symbols N, M, K are equal to the number in sequence along X, Y, and Z direction respectively (e.g. "sreg_x1y1z1" for the part in the back, bottom, left corner of the whole project).

Initial QW-Simulator MultiGPU's log window shows an ordered list of accessible GPU devices. User can adjust it in the GPU tab in Preferences dialogue. The devices will be attached to Subregions in the order specified in Preferences dialogue. This means that a particular device will handle all necessary computations for the region to which it is attached. All attachments are made and listed in the log window shortly after starting the simulation. After simulation finishes all devices are released. When all of the devices are already attached, the remaining Subregions will be computed on CPU, similarly to normal QW-Simulator.

Considering that the performance of QW-Simulator MultiGPU depends on the slowest Subregion, it is advised to use devices of similar capabilities and avoid pairing GPUs with CPU. Another performance factor worth consideration is synchronisation between Subregions. The time required for full exchange of the necessary data is proportional to the Subregion Border's area given in cells. Therefore for some particular projects it is beneficial to choose Subregion Border with minimal area. Another factor that user needs to acknowledge is the fact that memory allocated on the GPU still requires at least the same amount of memory allocated on the CPU, mostly for viewing purposes.