It was the first league defeat for Arsene Wenger's men since they lost to Liverpool on the opening day of the season and their first on the road since March.
More importantly, it handed Chelsea the chance to go six points clear at the top when they visit bottom club Sunderland on Wednesday.
It looked like business as usual for visitors, who have been scoring freely in recent weeks, when Alexis Sanchez netted his 12th league goal of the season with a deflected free kick after 20 minutes.
Everton though warmed to the task, levelling through Seamus Coleman's header a minute before halftime.
With the seconds ticking away towards the end, Williams rose high to nod Ross Barkley's corner into the net for his first Everton goal, securing the victories first win since the end of October.
It moved Everton up to seventh in the table and offered some cheer to manager Ronald Koeman.
"We showed unbelievable spirit to come back and we deserved the win," the Dutchman told BT Sport.
Everton struggled for confidence in the early stages as they retreated, inviting Arsenal to attack.
Francis Coquelin was brought down on the edge of the area and Sanchez stepped up to hit a tame free kick kick that looked to be covered by Maarten Stekelenburg until it deflected off Williams and beat the keeper's outstretched hand.
The goal put Sanchez joint top of the league's scoring charts with Chelsea's Diego Costa.
It also seemed strange to spark life into the hosts who gradually came out of their shells with Aaron Lennon slicing an effort wide before Coleman leveled with a glancing header from Leighton Baines' cross.
Mesut Ozil blasted a good chance over for Arsenal at the start of the second half but Everton were on top and pushing for the winner when Williams sent Goodison Park wild.
In a frenetic final, Everton defender Phil Jagielka was sent off for a second booking in the fourth minute of stoppage time.
Arsenal also had two attempts cleared off the line and somehow prevented the home team from grabbing a third goal when keeper Petr Cech was caught upfield.
Cristiano Ronaldo: I'd have more Ballon d'Or wins than Messi if we'd played in the same team
Fresh from winning his fourth Ballon d'Or on Monday evening, Cristiano Ronaldo has suggested that he'd have even more gongs than rival Lionel Messi if the pair had played in the same team.
The 2016 award sees the Real Madrid man move to within one of the Argentines on the all-time list, but despite trailing in the head-to-head, Ronaldo again reiterated his belief that he is a better all-round player than Messi.
"It would be interesting to see both of us in the same team. I think great players should play together. So if we were in the same team, I think I would have more than him, but he didn't want to be far off, "he told speakers.
"Everyone knows Messi is a great player. He's won five Ballon d'Ors."
Asked in the aftermath of the ceremony if he has any ambition of overhauling Messi to become the most decorated individual in the history of the award, the Los Blancos forward admitted that he will fight to achieve that very same goal in the coming seasons.
"I'm going to try. I'll be in the fight, as always," he said.
The pair will once again face off to win FIFA's Best Player award at a ceremony in Zurich on January 9, with Ronaldo the overwhelming favorite to scoop the new prize.
How does Cristiano Ronaldo stay the best?
Football's greatest individual duel has come to be viewed through the prism of its most prestigious individual award over the past eight years. Lionel Messi and Cristiano Ronaldo's personal rivalry is illustrated best by the number of times they have held the Ballon d'Or aloft. It is the defining image of their professional strife and on Monday Ronaldo added another picture to the reel.
Four fingers shown to indicate how many times he has now been crowned the best player in world football, there was no ceremony for the victor to showcase their prize this year, but the image of Ronaldo clutching dear the 2016 Ballon d'Or nonetheless circulated within moments of the announcement.
Having won both the Champions League with Real Madrid and Euro 2016 with Portugal, few could feasibly argue that the 31-year-old was not a worthy winner. A golden ball to cap a golden year.
Some might point to Luis Suarez's achievements over 2016, or Antoine Griezmann's performances for both Atletico Madrid and France, as a case to the contrary, but Ronaldo's Ballon d'Or win was a reward for how he has achieved greatness just as much as it was for achieving greatness itself.
Now into the twilight of his career, Ronaldo should be in decline. In many ways he is. He no longer runs as fast or for as long as he once did, games are taking more of a toll, injuries are lasting that little bit longer. But despite all this Ronaldo has found a way to stay at the forefront of world football. A way to remain the best.
Faced with his own mortality, Ronaldo adapted his game. He is now a master of maximization, only expending his energy where there is likely to be a reward. As a complete forward he plays the percentages, preserving himself while simultaneously imposing himself with as much impact as possible.
Compare the Portuguese to the spindly, erratic winger he was upon breaking through at Manchester United to what he is now and his transition becomes obvious.
Intelligence is a trait of Ronaldo's that often goes unrecognized, simply because there is so much more to marvel at. But make no mistake, the Real Madrid number seven is as smart as they come in football. He couldn't have been able to adapt and evolve in the way he has otherwise.
Ronaldo's sustained supremacy in spite of his slide into thirty-somethingdom is at odds with so many stars before him who have faded at the same stage of their careers. Take Kaka, for instance, the last player other than Messi or Ronaldo to win the Ballon d'Or. At 34 the Brazilian is a few years older than either of his contemporaries, but hasn't been considered among the very best for a long time.
In fact, Kaka's 2007 Ballon d'Or proved to be something of a watershed, with his career on the decline from that point on. He now plays for Orlando City in MLS.
Ronaldinho is another example of a glittering talent who scaled the highest heights only to fall as quickly as he had risen. By the time he left Barcelona to join AC Milan in 2008 he was as good as finished. Andriy Shevchenko, winner of the 2004 Ballon d'Or, failed to sustain his supremacy beyond his 30th birthday.
Even Messi has faded as the one-man match winner he once was. He too has adapted his game, becoming more of a playmaker over the past year or so, dropping deeper to provide the pass rather than getting on the end of it, but there are signs of the Argentine slowing down in a way not even Ronaldo has yet.
2016 may well go down as the year the Messi and Ronaldo duopoly was finally broken. The Barcelona and Real Madrid duo still occupied the top two spots in the Ballon d'Or voting, but for the first time in nearly a decade there was a genuine discussion to be had over who the best in the world really is.
Suggestions of others, like Luis Suarez or Antoine Griezmann, are no longer so farfetched, so outrageous.
But regardless of whether the Ballon d'Or is once again a close contest or not, Ronaldo deserves the acclaim for what he achieved in 2016. His year was about more than just European glory for club and country. His year was a tale of intelligence, adaptability and doing what it takes to stay at the top.
Emery: PSG will have chances to progress against Barcelona
Despite Paris Saint-Germain being handed a difficult last-16 tie against Barcelona, coach Unai Emery believes the French champions will get opportunities to reach the quarter-finals.
The Spanish tactician is no stranger to achieving success in European competition, having been at the helm at the perennial Europa League winning Sevilla.
Although his side are underdogs going into their clash against the Blaugrana, Emery believes that his side have an equally high chance of progressing as their opponents.
"We know that there were several options for us in the draw but at this stage of the competition, every team is of a very good standard," said the PSG boss.
"The important thing is that it is a game we want to play in. We are going to prepare at our best for two big games and we will have chances to progress.
"Either team could progress. I won't give a percentage chance of qualifying for either of us but I'm happy to be coming up against Barcelona."
Chelsea and Man City fined for fracas
Chelsea and Manchester City have been fined 100,000 pounds and 35,000 pounds respectively for failing to control their players during a Premier League encounter on Dec. 3, the FA said on Tuesday.
City striker Sergio Aguero's reckless tackle on Chelsea defender David Luiz in stoppage time sparked a melee at the Etihad Stadium, with players and coaching staff from both teams involved in ugly scenes.
Aguero and team mate Fernandinho were sent off and are serving four and three-game bans respectively.
"The two clubs were charged for failing to ensure their players conducted themselves in an orderly fashion and / or refrained from provocative behavior," the FA said in a statement.
League leaders Chelsea, who have claimed nine successive victories to top the table by three points from Arsenal, won the game 3-1.
Their fine is heavier than City's because of previous disciplinary indiscretions.
City have dropped off the pace in recent weeks and are seven points adrift of Chelsea in fourth place following Saturday's 4-2 defeat at champions Leicester City.