In all likelihood, if the system uses a proprietary form-factor, it probably won't be the actual physical dimensions (width, length) that are the problem but rather the placement of components on the backplane and of the standoff holes.
If the backplane is unusual, you can probably/hopefully just pop in a compatible bezel (figure 1), though the case may not be designed to support replacing the bezel (you may be able to just cut a hole).
If the standoff holes (figure 2) are in the wrong place, then you may have to find other means of securing the board to the case, or if you are lucky, perhaps secure it with just one or two screws.
FormFactors.org has a comprehensive list of form-factors for motherboards, power-supplies, and other components, with links to PDFs of the specifications for each that include dimensions and mounting-hole placements.
Figure 1: Variety of bezels:
Figure 2: Standoff holes:
Best Answer
Motherboards follow standard specifications.
These specifications include many different metrics and details, such as motherboard dimensions, features, etc.
Modern computer chassis (cases) are ATX standard. Most of the cases you will be looking at are ATX. However there are options around this.
Read 1
Fitting a motherboard into a case is for the most part straight forward. There are some things you will need to know.
Take these into account:
I/O shield.
The I/O shield / I/O ports are the ports that have the USB ports, the network port, the sound ports, and stuff like that. Each motherboard will come with a metal sheet that has holes that correspond to the ports for your motherboard. You will see in the case that it already has one, pop it out and use the one provided with the motherboard.
Make sure you install the I/O shield as one of the first steps, before putting the motherboard in. If you do not, you will have to remove the motherboard to insert it. Also pay attention to the "tabs" on the I/O shield -- these are supposed to touch the top of the connectors on the motherboard, to reduce EM emissions. Make sure you don't trap them against the case.
Risers.
All cases will come with "risers", they are little screw like things which make it so the motherboard safely secures to the case without grounding the motherboard.
Make sure you match up all the riser spots in the case with the motherboard holes. The hole positions on the case and motherboard are standardized. Then make sure each hole on the motherboard has the corresponding riser in place, so you can screw it down to them when the motherboard is in place. Some holes will not be necessary, this is why checking positions first is necessary.
Connect all power connectors on the motherboard, and reference the manual to make sure the ones you are plugging in are necessary. Also make sure to follow the "front IO" guide in the manual, this is so the power/reset/HDD LED are properly setup.
That's most of it. If you get stuck, look up howtos online to fill the gap.