Leigh Day and Co say they have been instructed by a number of local residents who are determined to stop the plans.
It was confirmed earlier on March 1 that the Isle of Wight Council will continue to run six of the Island's 11 libraries, but funding for the rest will stop in June, with hopes the community will take them on.
But campaigners say that what is left won't be comprehesive or efficient for all people and claim the council will be in breach of its duty in law, under section 7 of the Public Libraries and Museums Act 1964.
It is also claimed that there has been no analysis of the effect of library closures in areas of greatest deprivation, or the travel and parking costs at alternative libraries for those left without a local branch - as well as the impact on women and older and disabled people.
Richard Stein and Rosa Curling in the human rights department at Leigh Day & Co claim the IW Council's failed to properly consider the impact their decision will have on almost 20% of the current users, that this is unlawful and for this reason their clients have asked them to apply to the High Court to challenge the decision.