Can I hang my filter on one of the sides rather than on the back? I really don't have much room for any other kind of filter I have a 40 gallon tank

You can put your filter any place you like but you might need 2 depending on the filter and the length of your tank and 40's come 48 inches long or 30 inches long

Mines 48 inches should I also get a protein skimmer?

Protien Skimmers are NOT needed but I ALWAYS recomend them to be used. With enough LR, flow and a good CUC you shouldnt need any filter besides a Skimmer. What do you plan on stogking your tank with???

Just a few fish and some live rock. I know the just of what I need but what kind of filtration system do you recommend for my size tank

I run sump/fuges and a skimmer on mine with good flow. I havent run mechanical filtration on a SW/Reef tank for 10 years I think just LR with good flow, macro algea in my fuge and skimmer. I have seen to many times that mechanical filters tend to cause nitrate problems. I would just run the LR, good flow, good CUC and a hang on skimmer unless you can run a sump/fuge under you tank, or a gravity return fuge above your tank.

Sorry what's a cuc

sorry its a clean up crew as in an assortment of snail and hermits.

oh ok thanks so I shouldn't use a HOB filter?

Nothing wrong with a HOB filter on smaller tanks...under 75 gallons. I use AquaClears as a mater of fact when I dont use a sump on a RR tank. Orientation doesnt matter much as long as there is good water flow in the tank using the filters out put with the addition of power heads for added water movement.

Skimmers are a highly recommended item in more cases than not.

There are a ot of ways to have a great successful tank and very few sure fire ways to have a crappy one. But by the same token what works for me might not work quiet as well for you.

I would go with a skimmer as well.