Actually it does sort of make sense. The Schengen visa that should be procured and used should be issued by the country that is the entry point into the Schengen Zone and/or the main destination of the trip. Given the visa was issued by Norway but is not being used for a trip to or via Norway, there may be a problem when entering Schengen in Switzerland for a trip that doesn't involve Norway yet attempting to use a Norwegian visa. I've seen problems like this for Indian passport holders arriving into Sweden with a Swiss Schengen visa but not having any documentation to show that Sweden is just en route to Switzerland for the traveler. Eventually most are allowed in, but they can deny -- and sometimes have denied -- admission into the zone on the basis of having procured and trying to use a visa from a third country that is not on the itinerary.

If the visa were ever used for a trip to or via Norway as port of entry into the Schengen Zone, then there wouldn't be a problem in this regard.

By the way, when I first read the Swiss response, I thought they had messed up and reversed something; then I read it again and consulted some associates who work in this area and realized that my second read was accurate and they had not reversed things.

The OP should procure a Swiss visa or think about a change to the itinerary, even if there is a reasonable chance that the Swiss would allow entry into the Zone on a Norwegian visa -- but there is also a reasonable chance that the Swiss make a fuss about the Norwegian visa in such case where the visa is not being used, and has not been used, for travel to or via Norway.