Treebeard is bigger and seems quite stronger (throwing around boulders and such). That said, Groot has a healing factor and the capability to modify his size and shape (to a degree) to assist him in fighting.
I'd put my money behind Groot, since I feel like he's more adapted to fighting and has more options. But it's certainly possible that Treebeard is simply too big and tough for him to handle.
I'd go with Groot despite being smaller. His ability to spread out and create stabbing and whipping vines might be able to entangle the Ent's arms and break them off for a win.
But I could also see the Ent grabbing Groot and ripping him in twain.
Pretty much my thoughts. Groot's ability to weaponize his appendages (or create shields with them) should allow him to eventually whittle down the Ent. Plus he's faster.
Of course, that's assuming Treebeard isn't strong enough to simply grab hold of him and rip him apart.