Nobody wants to pinch and zoom to see your site
In this day and age of mobilegeddon everybody is one their phone either tweeting texting and most of all shopping. If you are a brand that has someing to sell and you do any business online then you know the importance of getting that sale.
If your website is not responsive then its safe to assume that your users will need to pinch and zoom to see your content. No body wants to pinch and zoom to read all your content or find a button to go to the next page. The average user will click away from your site within 15 seconds if your site doesn't load. and if your site is not designed for a mobile device, users reconigize that and click away. They will be recalling a previous frustrating experience they had with someone elses site that wasn't responsive. They wont want to go through that again. Being responsive gives users a better experience.