I understand why Tom Bombadil and the Barrowdowns were cut from the films. To most, they appear to be an add-on, useless and pointless. However, this could not be farther from the truth. They have a purpose, if somewhat obscure.
In the beginning, the hobbits are neither prepared nor mature enough to face the cruelties and evils of Middle Earth. Therefore, for much of the book, they rely on Guardians. First, when confronted with the Black Riders, they find the elves. Then Farmer Maggot. These are relatively weak and passing examples, but equal to the level of peril.
After leaving the Shire, the hobbits are alone in the Old Forest, which leads them to the Withywindle and Old man Willow. It is a danger none of them understand or grasp fully, with no clear solution or salvation. Thus, enters Tom Bombadil. Even after the knowledge of Bombadil is imparted and they’ve had their first tastes of danger, the hobbits are still caught by the Barrow-wight. Tom comes to save them again.
It is important to note, upon the parting of Tom and the Hobbits, their reluctance to be parted. They long for the safety, the security he offers. It was the same in the Shire. It is what they have always been used to: an easy life, with little fear. But now fear has found them and they do not know what to do.
Tom Bombadil is replaced by a long line of Guardians and caretakers: Aragorn, Glorfindel, Gandalf and the others of the Fellowship. Each watches out for the hobbits. Yet, in the end, the hobbits are separated momentarily from a guardian. Merry and Pippin are captured by the Uruk-hai. They manage quite well, finding and using courage they did not know they had. Frodo and Sam also no longer have a guardian and manage to navigate the Emyn Muil. However, again, they find guardians. Merry and Pippin meet Treebeard. Sam and Frodo meet Gollum. What is key to notice here, is the stature of these guardians. They grant more freedom and responsibility to the hobbits, especially in the case of Gollum.
Even so, the most important point to make is that the hobbits only have “guardians” until the moment(s) that really matter! Sam and Frodo are left alone in Shelob’s lair. Sam is left alone after Frodo is taken by the orcs of the tower. Pippin is alone against the madness of Denethor’s madness. Merry is alone to strike the Witchking. It is during these moments when the reader finally sees the true mettle and indomitable strength and courage of each hobbit.
The parting of Gandalf and the hobbits is the most important moment in the growth of Merry, Pippin, Sam and Frodo. Here, finally, they are forced to see the change in themselves; to stop relying on others and take up their burdens on their own, knowing they are now capable.
“’I am with you at present,’ said Gandalf, ‘but soon I shall not be. I am not coming to the Shire. You must settle its affairs yourselves; that is what you have been trained for. Do you not yet understand? My time is over: it is no longer my task to set things to rights, nor to help folk do so. And as for you, my dear friends, you will need no help. You are grown up now. Grown indeed very high; among the great you are, and I have no longer any fear at all for any of you.’” (974 Tolkien)
So Bombadil has a purpose, a very important one! He, and the other guardians, is there to guide and teach the hobbits. They could be seen as instructors, easing the hobbits towards self-reliance. The guardians are crucial to the plot. In part, Lord of the Rings is not just a story of good versus evil, epic battles, or virtue, but a story about growth.