About 700 people live around Camp Pharsalia Prison, which also sits just four miles from the closest school, Otselic Valley Central School.
Richard Pilec of Otselic also lives just a few miles away.
He moved his family from New York City.
"We moved out here to be comfortable and safe and to raise our family quietly in the right way and it just makes me nervous," said Pilec.
What makes Pilec nervous is the Governor’s plan to replace the prison with a facility to house up to 500 dangerous sex offenders after they are paroled.
It would also create up to 1,000 new jobs.
"It's no good for the neighborhood, for the children, for anybody," said Lucy Cuzzo of Norwich.
Cuzzo's daughter-in-law, a mother of three, agrees.
"I have to worry about my kids, like I have to be with them, side by side," said Jennifer Cuzzo of Norwich.
Assemblyman Clifford Crouch (R – Binghamton) is among the lawmakers who have been fighting to keep Camp Pharsalia open.
But he says the sex offender facility is a winning alternative.
"I think it will bode well for people wanting to live in Chenango County or close by. They're going to spend their income here like any other community," said Assemblyman Crouch.
Crouch hasn't seen the plans, but he says the facility should be very secure, with a fence around the grounds.
It will be a much different facility than the minimum security prison that sits there now.
The Governor will put the plan for Pharsalia in his 2006-2007 budget proposal.
It calls for spending $130 million to close and demolish the prison, and build the secure facility for sex offenders.
That facility would be run by the State Office of Mental Health.