The "Mary" that Percy Shelley is referring to is his wife and author Mary Shelley. She is most well known for her novel Frankenstein, but published many other books before her death in 1851.

Mary was the daughter of philosopher and political writer William Godwin and famed feminist Mary Wollstonecraft, but received no formal education. She taught herself mostly through her fathers library and the visits to her father from famous poets of the time, like Samuel Taylor Coleridge and William Wordsworth. Mary published her first poem in 1807.
Mary and Percy were married in 1816, two years before she published Frankenstein and 8 years before this poem was published. The dedication of this poem (To Mary (On Her Objecting to the Following Poem, Upon the Score of its Containing No Human Interest) indicates the close literary relationship the two had and the importance of each's opinion. Mary Shelley even edited the first edition this poem.
The rest of this passage seems to indicate that Percy disagrees with Mary's opinion that the poem "tell[s] no story", and even asks her to "this one time" accept the poem as simply a "visionary rhyme" and asks her to be content with that.