Many advanced MRI image acquisition and reconstruction methods see limited application due to high computational cost in MRI. For instance, iterative reconstruction algorithms (e.g. non-Cartesian k-space trajectory, or magnetic field inhomogeneity compensation) can improve image quality but suffer from low reconstruction speed. General-purpose computing on graphics processing units (GPU) have demonstrated significant performance speedups and cost reductions in science and engineering applications. In fact, GPU can offer significant speedup due to MRI parallelized-data structure, e.g. multi-shots, multi-coil, multi-slice, multi-time-point, etc. We propose an implementation of iterative MR image reconstruction with magnetic field inhomogeneity compensation on multi-GPUs. The MR image model is based on non-Cartesian trajectory (i.e.spiral) in k-space, and can compensate for both geometric distortion and some signal loss induced by susceptibility gradients.