One need not go to FINNEGANS WAKE for a circular structure but to VN's own short story "The Circle", which was published in 1936 and hence precedes Joyce's book of the night, a fact to which VN draws attention in his note on "The Circle" in A RUSSIAN BEAUTY AND OTHER STORIES.A. Bouazza.
The poem's starting line In my beginning is my end, which can be understood as a poetical expression of [T.S.Eliot's] desire that his ashes be kept there [ie from where his ancestors emigrated], is a derivation of Mary Queen of Scots' motto "En ma fin est mon commencement" ("In my end is my beginning"). [Wiki].
Charles