Hawk-Eye
Hawk-Eye is a computer vision system used to visually track the trajectory of a ball and display a profile of its statistically most likely path as a moving image. It is used in more than 20 major sports, including baseball, cricket, tennis, badminton, hurling, rugby union, soccer, Gaelic football, American football, and volleyball.
The Sony-owned Hawk-Eye system was developed in the United Kingdom by Paul Hawkins. The system was originally implemented in 2000 for television purposes in cricket. It works via the use of up to ten high-performance cameras, normally positioned on the underside of the stadium roof, which track the ball from different angles. The video from the cameras is then triangulated and combined to create a three-dimensional representation of the ball's trajectory. Although not infallible, Hawk-Eye is advertised to be accurate to within.
Hawk-Eye is increasingly used as an impartial review in sports, having been accepted by governing bodies in tennis, cricket, and association football as a means of adjudication. Hawk-Eye has been used for the Challenge System since 2006 in tennis and Decision Review System in cricket since 2009. The system is also used to determine whether the ball has crossed the goal line in football as a means of goal-line technology, implemented in the 2013–14 Premier League season and now present at many domestic leagues and international competitions.
Method of operation
All Hawk-Eye systems are based on the principles of triangulation using visual images and timing data provided by a number of high-speed video cameras located at different locations and angles around the area of play. For tennis there are 10 cameras. The system rapidly processes the video feeds from the cameras and ball tracker. A data store contains a predefined model of the playing area and includes data on the rules of the game.In each frame sent from each camera, the system identifies the group of pixels which corresponds to the image of the ball. It then calculates for each frame the position of the ball by comparing its position on at least two of the physically separate cameras at the same instant in time. A succession of frames builds up a record of the path along which the ball has travelled. It also "predicts" the future flight path of the ball and where it will interact with any of the playing area features already programmed into the database. The system can also interpret these interactions to decide infringements of the rules of the game.
The system generates a graphic image of the ball path and playing area, which means that information can be provided to judges, television viewers or coaching staff in near real-time.
The tracking system is combined with a back-end database and archiving capabilities so that it is possible to extract and analyse trends and statistics about individual players, games, ball-to-ball comparisons, etc.
History
Hawk-Eye was developed in 2000 by engineers at Roke Manor Research Limited, then a Siemens subsidiary in Romsey, England. Paul Hawkins and David Sherry submitted a United Kingdom patent application for the technology, but this was subsequently withdrawn. All of the technology and intellectual property was spun off into a separate company, Hawk-Eye Innovations Ltd, based in Winchester, Hampshire. This was initially established as a joint venture between Roke Manor Research and Sunset + Vine. At the time, Sunset + Vine produced the Channel 4 television cricket coverage where Hawk-Eye was first used.On 14 June 2006 a group of investors—led by the Wisden Group and that included Mark Getty, a member of the wealthy American family and business dynasty—bought the company for £4.4m. The acquisition was intended to strengthen Wisden's presence in cricket and allow it to enter tennis and other international sports, with Hawk-Eye working on implementing a system for basketball. According to Hawk-Eye's website, the system produces much more data than that shown on television.
Put up for sale in September 2010, it was sold as a complete entity to Japanese electronic giant Sony in March 2011 in a deal that valued the company at £15m-£20m.
Deployment in sport
Cricket
The technology was first used by Channel 4 during a Test match between England and Pakistan on Lord's Cricket Ground, on 21 May 2001. It is used by the majority of television networks to track the trajectory of balls in flight. Its major use in cricket broadcasting is in analysing leg before wicket decisions, where the likely path of the ball can be projected forward, through the batsman's legs, to see if it would have hit the stumps.In the winter season of 2008/2009 the ICC trialled a referral system where Hawk-Eye was used for referring decisions to the third umpire if a team disagreed with an LBW decision. Initially the third umpire was able to look at what the ball actually did up to the point when it hit the batsman, but could not look at the predicted flight of the ball after it hit the batsman. The third umpire is now able to see the projected path of the ball too, and Hawk-Eye is currently sanctioned in international cricket even though some doubts remain about its accuracy.
When an LBW decision is referred to Hawk-Eye, it assists in assessing against three criteria:
- Where the ball pitched
- The location of impact on the leg of the batsman
- The projected path of the ball past the batsman
Due to its real-time coverage of bowling speed, the system is also used to show delivery patterns of a bowler's behaviour such as line and length, or swing/turn information. At the end of an over, all six deliveries are often shown simultaneously to illustrate a bowler's variations, such as slower deliveries, bouncers and leg-cutters. A complete record of a bowler can also be shown over the course of a match.
Batsmen also benefit from the analysis of Hawk-Eye, as a record can be brought up of the deliveries from which a batsman scored. These are often shown as a 2-D silhouetted figure of a batsman and colour-coded dots of the balls faced by the batsman. Information such as the exact spot where the ball pitches or speed of the ball from the bowler's hand can also help in post-match analysis.
Tennis
In Serena Williams's quarter final loss to Jennifer Capriati at the 2004 US Open, three line calls went against Williams in the final set. Though the calls were not reversed, there was one overrule of a clearly incorrect line umpire call, by the chair umpire Mariana Alves, that the TV replay showed to be good. These errors prompted talks about line calling assistance especially as the Auto-Ref system was being tested by the U.S. Open at that time and was shown to be very accurate.In late 2006 Hawk-Eye was tested by the International Tennis Federation in New York City and was passed for professional use. Hawk-Eye reported that the New York tests involved 80 shots being measured by the ITF's high speed camera, a device similar to MacCAM. During an early test of the system at an exhibition tennis tournament in Australia, there was an instance when the tennis ball was shown as "Out", but the accompanying word was "In". This was explained to be an error in the way the tennis ball was shown on the graphical display as a circle, rather than as an ellipse. This was immediately corrected.
Hawk-Eye has been used in television coverage of several major tennis tournaments, including Wimbledon, the Queen's Club Championships, the Australian Open, the Davis Cup and the Tennis Masters Cup. The US Open Tennis Championship announced they would make official use of the technology for the 2006 US Open where each player receives two challenges per set. It is also used as part of a larger tennis simulation implemented by IBM called PointTracker.
The 2006 Hopman Cup in Perth, Western Australia, was the first elite-level tennis tournament where players were allowed to challenge point-ending line calls, which were then reviewed by the referees using Hawk-Eye technology. It used 10 cameras feeding information about ball position to the computers. Jamea Jackson was the first player to challenge a call using the system.
In March 2006, at the Nasdaq-100 Open in Key Biscayne, Florida, Hawk-Eye was used officially for the first time at a tennis tour event. Later that year, the US Open became the first grand-slam tournament to use the system during play, allowing players to challenge line calls.
The 2007 Australian Open was the next grand-slam event to implement Hawk-Eye in challenges to line calls, where each tennis player in Rod Laver Arena was allowed two incorrect challenges per set and one additional challenge should a tiebreak be played. In the event of an advantage final set, challenges were reset to two for each player every 12 games, i.e. 6-all, 12-all, etc. Controversies followed the event as at times Hawk-Eye produced erroneous output. In 2008, tennis players were allowed three incorrect challenges per set instead. Any leftover challenges did not carry over to the next set. Once, Amélie Mauresmo challenged a ball that was called in, and Hawk-Eye showed the ball was out by less than a millimetre, but the call was allowed to stand. As a result, the point was replayed and Mauresmo did not lose an incorrect challenge.
The Hawk-Eye technology used in the 2007 Dubai Tennis Championships had some minor controversies. Defending champion Rafael Nadal accused the system of incorrectly declaring an out ball to be in following his exit. The umpire had called a ball out; when Mikhail Youzhny challenged the decision, Hawk-Eye said it was in by. Youzhny said after that he himself thought the mark may have been wide but then offered that this kind of technology error could easily have been made by linesmen and umpires. Nadal could only shrug, saying that had this system been on clay, the mark would have clearly shown Hawk-Eye to be wrong.
The 2007 Wimbledon Championships also implemented the Hawk-Eye system as an officiating aid on Centre Court and Court 1, and each tennis player was allowed three incorrect challenges per set. If the set produced a tiebreak, each player was given an additional challenge. Additionally, in the event of a final set, where there is no tiebreak, each player's number of challenges was reset to three if the game score reached 6–6, and again at 12–12. Teymuraz Gabashvili, in his first round match against Roger Federer, made the first-ever Hawk-Eye challenge on Centre Court. Additionally, during the finals of Federer against Rafael Nadal, Nadal challenged a shot which was called out. Hawk-Eye showed the ball as in, just clipping the line. The reversal agitated Federer enough for him to request that the umpire turn off the Hawk-Eye technology for the remainder of the match.
In the 2009 Australian Open fourth round match between Roger Federer and Tomáš Berdych, Berdych challenged an out call. The Hawk-Eye system was not available when he challenged, likely due to a particularly pronounced shadow on the court. As a result, the original call stood.
In the 2009 Indian Wells Masters quarterfinals match between Ivan Ljubičić and Andy Murray, Murray challenged an out call. The Hawk-Eye system indicated that the ball landed on the centre of the line despite instant replay images showing that the ball was clearly out. It was later revealed that the Hawk-Eye system had mistakenly picked up the second bounce, which was on the line, instead of the first bounce of the ball. Immediately after the match, Murray apologised to Ljubicic for the call, and acknowledged that the point was out.
The Hawk-Eye system was developed as a replay system, originally for TV broadcast coverage. As such, it initially could not call ins and outs live.The representation of the trajectory results in terms of where the ball lands is called Shot Spot.
The Hawk-Eye Innovations website states that the system performs with an average error of. The standard diameter of a tennis ball is, equating to a 5% error relative to ball diameter. This is roughly equivalent to the fluff on the ball.
Hawk-Eye has developed a technology called 'Hawk-Eye Live', which uses the 10 cameras to call shots in or out in real time, with an 'out' call being signified by a speaker emitting an 'out' sound that emulates a human line judge. The technology was initially expected to be in place for the 2019 US Open. The 2021 Australian Open was the first Grand Slam tournament to use Hawk-Eye Live for all matches in place of line judges, in part to reduce personnel during the COVID-19 pandemic, followed by the US Open later that year. Previously, the 2020 US Open used Hawk-Eye in place of line judges for all matches except those held at Arthur Ashe Stadium and Louis Armstrong Stadium.
Clay court tournaments, notably the French Open, are generally free of Hawk-Eye technology due to marks left on the clay where the ball bounced to evidence a disputed line call. Chair umpires are then required to get out of their seat and examine the mark on court with the player by their side to discuss the chair umpire's decision. The 2021 Mutua Madrid Open became the first major tournament on clay to use an electronic system to check the bounce of the ball on the court when in doubt in the two main stadiums. The system was developed by FoxTenn, a Spanish company located in Barcelona. It uses real images of the ball captured by 40 cameras located at ground level, synchronized with lasers and working at up to 3,000 images per second to determine whether it has bounced in or out of the court's limits. This system was previously used at the 2020 Rio Open in Rio de Janeiro and at the 2021 MUSC Health Women's Open in Charleston, South Carolina.