Model your system as $\dot {\mathbf x} = f(\mathbf x, \mathbf u)$, where $\mathbf u$ is your IMU input. If it weren't for all the pesky rotations, you could model this as $\dot {\mathbf x} = \mathbf u$ (i.e., you're just integrating the IMU input).
In other words, model your system as something that gets rotation rate and acceleration "commands", and has a state vector (your starter state vector will have three each of velocity, position, and orientation, so it'll have 9 or -- if you use quaternions for rotation -- 10 states).
I'm pretty sure that a better way to represent this is by modeling the system as $\dot {\mathbf x} = g_{rot}(\mathbf x) \mathbf u$, where $g_{rot}(\mathbf x)$ just rotates the IMU command into the state vector's frame of reference.
In the case of never having some ground truth (such as a GPS position) to compare your filter to -- this will work, but you will find that your covariance will inexorably grow. The only way that such a system would be useful is if you could initialize it to some known state at start-up. Rather than having the covariance as a handy way of knowing how to set the gains when you do get a correction, it'll just be an indication to the user of how unreliable the data is.
Note that this is how "pure" inertial navigation works -- before the trip starts, the inertial nav system is started, and basically "told" "OK, you're sitting still at this point with respect to the Earth, and you're pointed this direction". Then everything is held steady for a while, then away you go.
Compared to any sort of reference signal fused with IMU data, the position estimate will degrade very quickly. "Navigation grade" IMUs are absurdly expensive (many thousands of $$) and the really good ones are typically export controlled, and all of this is for a reason. If you're using cell-phone quality IMUs, don't expect to hold an accurate position for very long at all.
I've done one GPS/IMU fusion effort; this was the approach I took and it worked well for me.