Yes, a window function just applies a weighting function to your data.
For N-D data you can view the window function as a combination of N 1-D windows which are all orthogonal to each other.
As the weights of the 1-D windows to not depend on the other dimensions you can either apply each separately or combine them to get a single N-D window.
e.g. for 3D (in matlab, it should be simple enough to translate to python)
hammx=hamming(L); %1D window
hamz=permute(hamx, [3, 2, 1]);
ham3=ones(L, L, L);
for i=1:100 % there is probably a much better way to compute this
ham3(i, j, :)=ham2(i, j).*hamz;