Westminster leader Stephen Flynn said the Government had to listen to the public and remove the dukedom from the prince.