In Chamber of Secrets, Tom Riddle mocks the assistance Harry receives at the start of their final confrontation - Fawkes and the Sorting Hat arriving - as pathetic.

So this is what Dumbledore sends his great defender. A songbird and an
old hat.

It's established elsewhere in the series, however, that Riddle was a top student, and that he was incredibly well versed in lore related to the Hogwarts founders. Wouldn't he have known exactly how valuable a phoenix could be, in that situation? And how potentially dangerous the hat was to his plans?

Tom / Voldemort is incredibly arrogant, of course. But this seems hard to explain even with arrogance. Wouldn't he have killed Fawkes and destroyed the hat before dealing with Harry, who didn't even have a wand at that point?

Neither Fawkes nor the hat appeared to be a threat.

Phoenixes are not known for their offensive prowess. There wasn't really anything that Fawkes could do to harm Tom Riddle. At most, Fawkes would be able to heal Harry if he got injured, but Riddle was planning on killing Harry rather than injuring him.

As for the hat, prior to this event there does not seem to be any record of the hat doing anything other than sorting students. As such, Riddle had no reason to think that it was anything other than a useless hat.

Since Riddle had nothing to fear from Fawkes or the hat, he had no reason to deal with them. He had a wand and a basilisk and he was up against an unarmed second-year student. He could well afford to play around. In fact, Riddle probably wanted Harry to have these items to help him, as that would be a better display of Riddle's superiority. Note what he says right before calling the basilisk:

"Now, Harry, I'm going to teach you a little lesson. Let's match the powers of Lord Voldemort, Heir of Salazar Slytherin, against famous Harry Potter, and the best weapons Dumbledore can give him...."

This is quite similar to what he did with Harry at the end of Goblet of Fire, when he gave Harry his wand. He didn't want to simply kill Harry; he wanted to demonstrate his superiority over Harry. As he said then:

I will give him his chance. He will be allowed to fight, and you will be left in no doubt which of us is the stronger.

That Fawkes ended up blinding the basilisk was not the biggest deal, as the basilisk could still kill with its fangs. Once the basilisk struck Harry, Riddle probably should have kept Fawkes away from him, but as Bellatrix pointed out he wasn't thinking of the fact that phoenixes have healing powers. And even after Harry was healed there still was no real issue because Riddle still had a wand with which he could kill Harry, while Harry only had a sword.

The real mistake was leaving the diary lying around where it could easily be destroyed, but that too only became a problem because Harry unexpectedly pulled a sword out of the hat which ended up getting him access to a basilisk fang with which to destroy the diary.

Perhaps Voldemort should have respected Dumbledore enough to think that there might be some purpose to an apparently useless hat, but that's where the arrogance factor comes in.

Tom forgot phoenix tears can heal.

Tom Riddle didn’t remember that phoenix tears could heal at the time he was confronting Harry. He says himself when he realized what had happened that he forgot that.

“Phoenix tears …’ said Riddle quietly, staring at Harry’s arm. ‘Of course … healing powers … I forgot …”
_{^{- Harry Potter and the Chamber of Secrets, Chapter 17 (The Heir of Slytherin)}}

It’s pretty obvious, even aside from his own admission, that Tom forgot that phoenix tears can heal. He watches Fawkes start crying while approaching Harry, doesn’t try to move Fawkes away from him, and still considered Harry as good as dead.

“You’re dead, Harry Potter,’ said Riddle’s voice above him. ‘Dead. Even Dumbledore’s bird knows it. Do you see what he’s doing, Potter? He’s crying.’

Harry blinked. Fawkes’s head slid in and out of focus. Thick, pearly tears were trickling down the glossy feathers.

‘I’m going to sit here and watch you die, Harry Potter. Take your time. I’m in no hurry.”
_{^{- Harry Potter and the Chamber of Secrets, Chapter 17 (The Heir of Slytherin)}}

Since Tom didn’t remember about their healing tears, he’d have no other reason to consider a phoenix any threat. As Newt Scamander says in Fantastic Beasts and Where to Find Them, phoenixes have never been known to kill, and nothing about them seems like an obvious threat to Tom’s victory over Harry.

“PHOENIX

M.O.M. Classification: XXXX¹¹

The phoenix is a magnificent, swan-sized, scarlet bird with a long golden tail, beak, and talons. It nests on mountain peaks and is found in Egypt, India, and China. The phoenix lives to an immense age as it can regenerate, bursting into flames when its body begins to fail and rising again from the ashes as a chick. The phoenix is a gentle creature that has never been known to kill and eats only herbs. Like the Diricawl (see above), it can disappear and reappear at will. Phoenix song is magical; it is reputed to increase the courage of the pure of heart and to strike fear into the hearts of the impure. Phoenix tears have powerful healing properties.

¹¹ The phoenix gains a XXXX rating not because it is aggressive, but because very few wizards have ever succeeded in domesticating it.”
_{^{- Fantastic Beasts and Where to Find Them}}

He also didn’t seem to know the Hat’s use.

As for why he didn’t think the Sorting Hat was more of a threat, it doesn’t seem like he’s aware at the time that true Gryffindors can pull the Sword of Gryffindor out of the Hat. Upon recognizing the Hat, Tom’s reaction is to mock Harry and ask him if he feels brave and if he feels safe. If Tom thought Harry could get a sword from the Hat, even if he didn’t consider a sword to be a good match against a wand, it’s unlikely he’d mock him in that way.

“And that –’ said Riddle, now eyeing the ragged thing that Fawkes had dropped, ‘that’s the old school Sorting Hat.’

So it was. Patched, frayed and dirty, the Hat lay motionless at Harry’s feet.

Riddle began to laugh again. He laughed so hard that the dark chamber rang with it, as though ten Riddles were laughing at once.

‘This is what Dumbledore sends his defender! A songbird and an old hat! Do you feel brave, Harry Potter? Do you feel safe now?’

Harry didn’t answer. He might not see what use Fawkes or the Sorting Hat were, but he was no longer alone, and he waited with mounting courage for Riddle to stop laughing.”
_{^{- Harry Potter and the Chamber of Secrets, Chapter 17 (The Heir of Slytherin)}}

Also, Harry, who’d spent around two years as a Gryffindor, didn’t know the Hat could give him a sword. Tom, though he was four years older than Harry and more studious, wouldn’t necessarily know about that, either. Though it was known to some in the wizarding world, it seemed to be something that happened long ago, and was more like a legend. Rufus Scrimgeour says “reliable historical sources” say the Sword may appear in the Hat for worthy Gryffindors, which implies it’s been quite a long time since anyone pulled the Sword from the Hat.

“It belongs to Harry!’ said Hermione hotly. ‘It chose him, he was the one who found it, it came to him out of the Sorting Hat –’

‘According to reliable historical sources, the sword may present itself to any worthy Gryffindor,’ said Scrimgeour.”
_{^{- Harry Potter and the Deathly Hallows, Chapter 7 (The Will of Albus Dumbledore)}}

Though Tom studied and was knowledgeable about the founders, that doesn’t mean he’d have to know that Gryffindors can pull the Sword of Gryffindor from the Sorting Hat. Even if he’d come across that somewhere in his reading, he wouldn’t necessarily remember it. Hermione read about the Chamber of Secrets in Hogwarts: A History, but she couldn’t remember what it was.

“I wish I hadn’t left my copy at home, but I couldn’t fit it in my trunk with all the Lockhart books.’

‘Why do you want it?’ said Harry.

‘The same reason everyone else wants it,’ said Hermione, ‘to read up on the legend of the Chamber of Secrets.’

‘What’s that?’ said Harry quickly.

‘That’s just it. I can’t remember,’ said Hermione, biting her lip. ‘And I can’t find the story anywhere else –”
_{^{- Harry Potter and the Chamber of Secrets, Chapter 9 (The Writing on the Wall)}}

Studying something doesn’t mean remembering every piece of information about it, and there’s nothing about that particular story that’d make Tom remember that specifically. Though he was interested in the founders, it’s likely he wouldn’t take much note of a story about the Sorting Hat that’s not even about his house.

Even after all his years of studying, as an adult, he doesn’t seem at all concerned about the Sorting Hat’s presence. When he thought Harry was dead, he summoned the Sorting Hat himself, then put it on Neville’s head, though Neville is a Gryffindor.

“Still watching through his lashes, Harry saw Voldemort wave his wand. Seconds later, out of one of the castle’s shattered windows, something that looked like a misshapen bird flew through the half-light and landed in Voldemort’s hand. He shook the mildewed object by its pointed end and it dangled, empty and ragged: the Sorting Hat.

‘There will be no more Sorting at Hogwarts School,’ said Voldemort. ‘There will be no more houses. The emblem, shield and colours of my noble ancestor, Salazar Slytherin, will suffice for everyone, won’t they, Neville Longbottom?’

He pointed his wand at Neville, who grew rigid and still, then forced the Hat on to Neville’s head, so that it slipped down below his eyes.”
_{^{- Harry Potter and the Deathly Hallows, Chapter 36 (The Flaw in the Plan)}}

Therefore it’s not impossible that Tom Riddle, who was very interested in the founders, could not know about the. It’s not clear if he did actually know. However, it’s clear from his actions even as an adult that he didn’t consider the Sorting Hat being present a threat in any way.

Voldemort's age in the diary

I believe also that the piece of his soul when placed in the diary was only 16 and thus did not have the complete knowledge that Voldemort would have acquired by the time he was at the peak of his power.

According to HP LEXICON:

Tom Riddle says that he placed his "sixteen-year-old self" into the
diary ([CS17]), so he must have done it in the fall after the summer
when he killed the Riddles and before he turned 17 at the end of
December. His conversation with Slughorn where he learned about
Horcruxes must have taken place after school started that September,
so the timeframe for the creation of the diary Horcrux is between
September and December, 1943

According to his Pottermore profile, Tom Marvolo Riddle was born on 31 December, 1926. This means he would have been nearing his 55th birthday on the night he failed to kill Harry. Between his 16th birthday and his 55th he would have continued learning about both Hogwarts and its founders. He was not at peak power or knowledge at the age of 16. Thus gaps in his knowledge are to be expected.