Summary

Broadcasting体育节目目前是很常见的，因此，high-quality 和 powerful video是高度需求的。但是，通常camrea operator很难对一个dynamic object保持track camera's direction, 例如a particular player, a ball, 等等。 在这种case下， shootable method 局限于在a wide angle of view情况下，缓慢的move camera's gaze，或者基于prediction并不准确的control the gaze，而容忍shot不是非常well的情况。Super slow and close-up videos of remarkable player 或者 ball 被认为是quite valuable的，但是camera operator还不能够做这些工作。

为了solve the issue，我们developed " 1ms Auto Pan-tilt" technology. 这个technology可以自动控制camera's pan-tilt angles, 保证 object 位于视场的中心。类似autofocus 会keep an object in focus。甚至 high-speed object像bouncing pingpong ball in play 都可以通过 optical gaze controller和1000 fps high-speed vision 被 tracked at center.  The saccade mirror 控制 camera direction 不是通过moving camera itself，而是rotate two-axis small galvanometer mirrors. 它控制gaze by 60 degree， the widest angle，for both pan and tilt。但是steering the gaze by 40 degree，只需3.5ms。最新的prototype system获取a full HD image quality。1ms Auto Pan-Tilt movie的图像序列如图所示，可以看出the ball in the game总是位于each image的中心。我们envision the sytem for broadcasting sports game，但是still期待record detail dynamic of flying bird。

Abstract

我们develop一种叫做"Saccade Mirror"的optical high-speed gaze controller. 通过该controller和high-speed image processor，realize a high-speed pan/tilt camera。通常来讲，pan/tilt camera的gaze是由rotational actuator机械控制的。但是，因为pan/tilt cameras are exptected to use high speed image processing，sufficently high-speed performance不能通过usual method来获得。在我们的系统中，the camera itself是固定的，一个external Saccade Mirror subsystem被用做optical gaze control。一个任意的不超过30度的gaze direction 可以在不超过3.5ms的时间内实现。

Introduction

该研究的目的是realize a high-speed pan/tilt camera，实现change gaze direction extremely quickly，同时提供足够的angle of view。The term gaze 意思是the direction in which the images are captured by the camera. High speed image processor指的是超过1000 frames per second的camera。 超过80%的sensory information是由visual organ获得的。对于robots，visual servo system已经被widely used来grasp changing external environment。拥有30 fps的Vision sensor，例如CCDs，主要用于visual information processing. 但是， 这种方法不能实现high-speed visual control, 因为image sampling rate局限于30Hz。在本例中，a high-speed image processor被证明非常的useful。

另外一方面，在一些application, it is necessary to acquire images over a range wider than the camera's original angle of view. 因此，可以控制gaze的camera，可以被称为pan/tilt cameras，或者叫active vision system，已经广泛的应用于monitoring，inspection。通常情况下的 pan/tilt camera是挂载在two-axis的rotational platform，including acturators。如果一个high-speed image processor 可以以足够高速来控制 gaze， 非常多的application可以被实现。例子包括：Observation of high-speed dynamic object without motion blur. Quasi-extension of the angle of view by combining multiple images taken from different gaze direction in real time. 这种system在很多vision application，包括robot vision，medical service，and broadcasting。但是，sufficiently high-speed performance 不能通过rotational actuators来机械性的获得。The time required to reach the desired gaze direction通常要超过20ms，而imaging cycle往往不超过1ms。这是一个critical bottlenect来实现一个high-speed pan/tilt camera。为了解决这个问题，我们开发了一个external high-speed gaze controller, 被称为Saccade Mirror。这个名字是基于快速的movement of human eye, which is well known as Saccade. 两轴的rotational mirrors，the critical part of the system，在millisecond-order快速的control the gaze direction.

Components of Saccade Mirror

两轴的rotational mirrors for high-speed gaze control.已经拥有several studies，目的是用conventional structure来realize a high-speed pan/tilt camera。但是，the required level of performance已经被obtained。 举个例子，Y. Nakabo 已经develop了一个1 ms的target tracking system，这是一个good example of a pan/tilt camera with high-speed image processor. 在这个系统中，一个high speed image procesor挂载在一个两轴的rotational platfrom, including high-speed actuator。  尽管这样，response time for switching the gaze direction 会超过20ms, 远远超过1ms的control cycle period。同时，the cutoff frequency of gaze control in the visual feedback is approximately 10Hz，而image rate is 1KHz.  For millisecond-order control，旋转的惯性必须尽可能的减少。

我们focused attention on 两轴的rotational mirrors for gaze control。在这个实例中，因为the rotating parts are two mirrors, the inertia of that subsystem can be substantially reduced. 但是two-axis rotational mirror还没有被用来作为pan-tilt camera。它们之前主要用来scan non-diverging bundles of rays, 比如laser beam. 举个例子，J.Taylor 和 H. Kim 使用camera, laser, 和两轴的rotational mirror开发了各自的depth measurement system。如果我们利用rotational mirrors来实现 pan/tilt camrea, one critical issue是the obtained angle of view是明显的限制的，因为mirrors 会很小并且rotate on two axes来构建图像。系统geometry的model如下。 Pinhole camera model被采用，angle of view 定义为给定camera scene的 angular extent。Rays from scene会汇聚于pinhole。 The bundle of ray汇聚的点叫做pupil。S(d) 是一个certain image plane的area，d是到pupil中心的距离。对于gaze control with mirrors，如果需要a wide angle of view，large mirrors仍旧是必须的。因此，不太容易去实现 both a wide angle of view 和 high speed performance at the same time because large mirrors are difficult to drive quickly. 这在使用independent two-axis mirrors的情况下，甚至更为困难。 我们称这个靠近pupil的mirror为pan mirror， 而另外一个是tilt mirror。如果大范围的controllable gaze ranges需要实现，tilt-mirror的size会比pan-mirror的size更大。

Pupil shift lenses for achieving practical angle of view

我们通过使用positive lenses来solve刚才的issue，或者称为“pupil shift lenses"。 Pupil shift lens的功能如图所示。A bundle of rays 可以被 refracted 通过 positive lens。Some bundle of rays concentrate at B but also concentrated at A. 把A当作camera的pupil，包括 positive lens 的 vision systems 拥有两个 pupils， the original one A 和 the shifted on B. 这个理论可以应用为 Saccade Mirror，使得 a wide angle of view和high-speed performance能够同时实现。如果我们需要在一个很高的速度进行gaze control，这种 combination应该会给出一个 practical angle of view。或者如果我们需要一个practical angle of view，这个combination会给出足够的高速。举个例子，为了获得30 degree angle of view without pupil shift lenses，the inertia of the tilt-mirror估计要比使用pupil shift lenses的vision system 大 100倍。

Prototype Saccade Mirror

在这个section，我们describe一个使用commercially available device 构成的 prototype Saccade Mirror。 Optical Setup，对于高速的gaze control，我们使用原先用于laser scanning的galvanometer mirror来做rotational mirrors。而对pupil shift lenses, 我们使用 three achromatic lenses。The setup（setup 就是设备、设置） 如图所示。光学的specification 如下所示：Gazing angle range大概是正负三十度，Maximum beam aperture 是 30mm，Maximum angle of view 是 38.6 deg，40 deg。The response time for switching the gaze from a certain direction to any desired direction was measured high-speed camera. 尽管当the mirrors很快速的scanned，the desired image captured by the high-speed camera看起来非常stable。The response time只有3.5ms. Compared with general mechanical pan/tilt camera(response time大于20ms），我们成功的achieve the expected millisecond-order response。

Setup for Target Tracking

我们开发了一个millisecond-order high-speed pan/tilt camera，其包含了一个prototype Saccade Mirror和一个high-speed image processor。我使用 camera来实现target tracking application。 The tracking algorithm 描述如下。一个在certain time捕获的image会传输到PC。 接下来，the target通过使用adequate thresholding，精确的与background进行了区分。然后，the mirror通过调整减少target与image center的距离为0。这个 process 每隔1ms来实现target tracking。

Frequency Response

The frequency response of the target tracking system是通过把一个marker附在rotating fan上作为target。The cutoff frequence大概在100Hz（pan gazing) 或者 超过100 Hz（tilt gazing)。The phase delay 大概是 180 degree （5 ms)。在一个previous study中， the frequency response of a target tracking system with mechanical pan/tilt camera was obtained. 这个cutoff frequency大概是10 Hz （pan gazing)，而phase delay at the cutoff frequency was 90 degree(25ms)。因此，compared with the previous system，我们的target tracking system成功的获得了超过10倍以上的performance level， in terms of cutoff frequency。However， the phase delay at cutoff frequency in our system was larger than that in the previous system。原因分析如下。如果使用windows PC， 一个2-3ms的delay会不可避免的产生，不考虑gazing method。 这个delay实际上不同于conventional system，whose response time 大概有20ms。但是，因为the response time for our millisecond-order pan/tilt camera大概是3.5 ms，由PC产生的2-3ms的delay变得相对large.

Observation of Moving Ball

我们的millisecond-order pan/tilt camera可能会非常usefull，来observe或者evaluate high-speed dynamic phenomena。如果一个high-speed camera可以被使用，the entire range of dynamic motion一定要让visual field of camera使用wide-angle lens。因此，the resolution of the desired area会变得很低。除此之外，motion blur 会因为target move at high speed而出现motion blur。我们focus on application in the field of sports。像篮球，足球，和Tennis非常难于连续的observe，因为the velocity vectors of balls 在运动中fluctuate非常快。获取fast-moving ball的动态图像，会make contribution to broadcasting techniques等。因此，作为一个exploratory experiment，我们使用millisecond-order pan/tilt camera来观dynamic rubber ball，the target suddenly came into the visual field，bounced against a table, and was finally hit by a racket。为了keep the images of the target ball stationary，they are corrected so that the center of mass in each image is placed at the center. 尽管the instant where the conventially 很困难去 capture，clearly observable sequence可能会成功的获得。球的估计trajectory可以很好的获得。

Conclusion

这篇文章中，我们提出了一个optical high-speed gaze controller，叫做Saccade Mirror。The developed prototype成功的演示了低至3.5ms。我们同时develop了一个millisecond-order pan/tilt camera，使用了这个prototype和high-speed image processor，我使用它来实现一个target tracking application。性能（主要是cutoff frequency)差不多是超过传统的convential mechanical pan/tilt camera十倍higher。

Reference

http://www.k2.t.u-tokyo.ac.jp/mvf/SaccadeMirrorFullHD/index-e.html