The mortgage lender will require you at some point to set up your Homeowner's Insurance, so might as well get started on it! For most people, buying a home is easily the biggest purchase you will ever make and you want to make sure its properly insured.
Just like with mortgage lenders, I recommend shopping around a bit because rates can very dramatically between companies. I recommend starting first with whomever insures your car and get a quote from them first - as likely you will get a discount for a multiple policy through them. Have your homeowners insurance policy start the day Before we close. That way if something happens, we know you're covered.
You can also get the utilities transferred over into your name. Depending on the location, you will have water, sewer, garbage (typically these are changed/billed by the city) and gas, power, and cable. Have everything start in your name on the day of closing.