Development and Integration of a Workpiece-Based Calibration Method for an Optical Assistance System

Abstract

Assistance systems utilize a broad range of technologies to provide information and guidance to workers in manufacturing. The use of light projectors, as of today, has seldom been catalogued in the relevant literature, and the implementation of such is yet to be found in production environments. However, light projectors may offer a cost effective enhancement for production processes, especially within the context of large-scale workpieces. Of the pertaining literature, only one calibration algorithm is currently considered applicable, thus acting as a framework of motivation for this paper. A novel calibration algorithm based on Newton’s method is presented and validated in conjunction with a proof-of-concept demonstration of the resulting accuracy, as well as the integration of such into an interface based on Node-RED, with MQTT as the main protocol.

1. Introduction

1.1. Motivation

Growing product variance stemming from an increase in customer requirements raises the demands on both the flexibility of production systems and the skills of employees on the shop floor [1]. Where approaches of Industry 4.0 are focused on the efficient automation and interconnection of systems providing the technical foundations for adaptive production, its successor, Industry 5.0, has evolved around notions that place the human at the center. The envisioned result of both trends are highly connected and human-oriented factories, in which the technologies are tailored to the needs of the people who work in them, and the latter, as the central decision-maker, can control production in unison with a multitude of autonomous systems [2].

The basic enablers of this vision are Human–Machine Interfaces (HMIs), which allow the interweaving of humans and cyber-physical systems. Individual interfaces provide a communication channel to transmit information with the help of a specific modality [3]. The modality of information transmission is characterized by basic human abilities to absorb information, whereby visual, acoustic and haptic exchange are the most frequently used. Based on these interfaces, numerous research projects in the context of smart production systems have dealt with the support of humans in complex activities under the keyword “Cognitive Assistance Systems” (CASs) [4].

The relevance of such systems increases in principle with the proportion of manual processes and with the product variance within a domain. This applies in particular to aircraft manufacturing, where the proportion of manual processes is substantial, with lot sizes within certain subsectors (e.g., cabin interior) approaching 1 [5]. Therefore, supporting manufacturing personnel in process execution is of central importance for time-efficient, robust and error-free aircraft production.

The domain of aircraft manufacturing poses special requirements that can also be found in the manufacturing of other large-scale products—namely, the vast quantity of assembly components that constitute an aircraft, as well as the pertaining range of dimensions of said components. From this, specific requirements can be derived for the coverage of the working areas of CASs. In addition, various processes are carried out simultaneously in a large workspace, occasionally in ergonomically unfavorable positions. For projection systems, this can lead to occlusions caused by the employees and tools on the shop floor, thus hindering information transmission in the assisted processes. Overall, the working environment can be described as challenging for the use of CASs.

1.2. Research Gap

However, most of the research on CAS focuses on small-scale production environments (e.g., assembly work benches) [6], hindering the direct applicability of the current approaches. Nevertheless, some of the core approaches of CASs with visual information output are generally applicable. These include augmented reality glasses, lasers, and video and light projectors (moving head spots). In view of the requirements described above, light projectors may be a cost-effective solution and are fundamentally suitable for use in aircraft production (note: more information about pricing of selected devices can be found in Appendix B). Nevertheless, there is not a significant number of papers on light projectors in the literature, which is due to the fact that they are mainly used in event technology, and no applications in the manufacturing industry are known. What exists are only a few papers touching on this technology’s potential as an information channel for a CAS [7,8]. So far, the integration of the moving head spot in the industrial laboratory environment has been demonstrated in principle. The commissioning, which includes the calibration of the device, has been solved using a frame so that a projection within a previously measured two-dimensional plane is enabled. Thus, there is a lack of a calibration methodology for three-dimensional space and, on the other hand, a methodology that does not require additional mechanical calibration jigs. To overcome this deficit, this paper presents a workpiece-based calibration method for moving head spots, which calculates the transformation between the moving head spot and a given reference coordinate system using Newton’s method. This not only simplifies the calibration process, but also makes it easier to work in three-dimensional space, enabling the illumination of curved objects such as those found in aircraft production.

1.3. Outline of This Work

Following the introduction, in Section 2 of this paper, the current state of the art is analyzed, underlining the framework of motivation. Section 3 explains the fundamentals of controlling a moving head spot and introduces the aircraft workpiece used for development and testing. Section 4 introduces the proposed calibration algorithm, which constitutes the core of this work. Following the calibration algorithm, the different aspects of the overall calibration process are validated (Section 5) and discussed (Section 6). The last two parts consist of Section 7, which summarizes the main contributions of this work, and Section 8, which motivates further research into the varying aspects of using a moving head spot as a CAS.

2. State of the Art and Related Work

This section provides an overview of pertinent state-of-the-art assistance systems in the field of manufacturing (Section 2.1), as well as related work on light projector commissioning (Section 2.2) and referencing based on component information (Section 2.3).

2.1. Assistance Systems

2.1.1. Classification of Assistance Systems in Manufacturing

Figure 1 serves as an overview for current research approaches in the field of assistance systems in manufacturing. The classification is based on the three criteria shown: the type of assistance (1), the modality used (2), and the technology leveraged in the form of the output device (3). The focus lies in cognitive assistance systems with visual information transfer, and these are marked in blue. Further detailed considerations of other types of assistance systems or alternative modalities within cognitive assistance systems will be omitted, as they are not relevant to the present work (marked in gray as to classify this work in the overall research field).

The illustrated classification method for assistance systems represents a combination of criteria commonly used within the pertinent literature, and is explained in more detail here. On the first level, a distinction is made regarding the type of assistance. More on this basic classification can be found in the work of [4,9]. The general aim of CASs is to support employees in their decision-making, or otherwise, in the execution of work processes with the aid of information, instructions, or feedback, thus imparting relief from mental fatigue. A detailed analysis of this type of assistance system in the context of manufacturing will be part of Section 2.1.2. Sensorial Assistance is intended to assist the worker by extending their capabilities of either acquiring information or focusing their attention on a specific piece of information [10]. Relevant instances often work with systems (mostly cameras) that record and document conditions such as product status [11], or track employee motions [12]. It should be noted that within the relevant literature, the terms “cognitive” and “sensorial” assistance are not always used in mutual distinction; in many systems, information acquisition, its processing, and its presentation to the human accompany each other, often rendering stringent distinction unnecessary. In several publications, therefore, the terms “Digital Assistance Systems” [13] or “Informational Assistance Systems” [14] can be found, in which the goal of mental relief is achieved by combining both approaches. Dissimilar to the first two types, Physical Assistance Systems provide not mental, but physical relief. This is realized, for example, by exoskeletons [15] that support specific body regions as a wearable suit, or by cobots that assume particularly unergonomic tasks in a common workflow [16].

The second level of classification relates only to cognitive assistance systems and is based on the modalities of information transfer in the work of [17,18]. They are derived from the human senses that allow us to perceive information. The most common modalities in which information can be transferred are visual, auditory and haptic. In this work, we focus on cognitive assistance systems with a visual information output that is focused separately. Examples for the the auditory modality can be found in [19], in which the worker is provided corresponding steps by a voice output on an Augmented Reality Glass, enabling hands-free work execution. Haptic information outputs are often realized in manufacturing by using vibration functionalities of a smart watch; e.g., [20] leverages this as a means to convey direct feedback about the quality of the conducted processes.

The third classification level pertains to the type of output devices selected and is also proposed as a classification criterion in similar approaches by [9,21]. In this context, it should be noted that some devices are not only capable of transmitting visual information, but are also capable of serving other modalities. These connecting lines are not drawn in the figure for simplicity. As depicted, the devices currently used in the literature range from fixed projection-based solutions to wearable devices. Based on the individual requirements of the use case, the developed assistance systems may consist of single or multiple components for information provision [21]. In the following subsection, we will examine the recent research in the field of CASs with visual output.

2.1.2. Cognitive Assistance Systems

With the primary objective of mental relief for the employee, the field of CASs in manufacturing is primarily concerned with the context-sensitive provision of process-relevant information during the assembly or disassembly of variant-rich products down to batch size 1 [22]. Within this context, CASs are often combined with physical assistance systems (e.g., cobots in human–robot collaboration [5,12]), or sensorial assistance systems, as mentioned in Section 2.1.1. In addition to assembly, research approaches in the field of maintenance [23], repair [24], overhaul [25] and logistics [26] can also be found. From an economic perspective, CASs may help to ensure an error-free and efficient process execution, thus saving time and additional rework costs.

Examining the possible output devices shown in Figure 1, different stationary projection-based approaches utilizing light [7], laser [11,27], and video projectors [22,28,29] as a component within their assistance systems can be found in the respectively cited literature.

A shared characteristic of these approaches is their limited scope of application due to the coverage of the devices, rendering them as non-directly transferable to components with larger dimensions. Given this constraint, these applications are often installed at spatially limited workstations (such as assembly tables). In order to cover larger work areas, additional devices must be purchased, increasing cost and control efforts. In addition, projection-based systems are always susceptible to occlusions. Therefore, the devices are typically ceiling-mounted directly over the component to avoid employee coverage of the projection surface. However, if the position of the projector cannot be oriented in such a way that occlusions can be avoided, this problem can also be countered with multiple devices, all with varying positions and projection angles.

Solutions based on mobile devices such as augmented reality glasses, with the HoloLens as a market-ready solution, allow the coverage of larger workspaces as well as a more immersive display of information due to their location flexibility, as shown in [30,31]. Nevertheless, localizing the AR glasses within space is a challenging task. Equipping the manufacturing and assembly environment with stationary reference markers (ArUco markers, April tags) for localization is infeasible within some domains, such as aircraft manufacturing, due to the requirement for installation, calibration and residue-free removal. Here, model-based (CAD) and feature-based (SLAM) methods represent an alternative for localization. However, the latter two methods have limitations in terms of accuracy in large-scale work environments [32]. Another as yet unsolved problem of Augmented Reality devices is “Simulator Sickness”, which describes a form of motion sickness during the prolonged use of immersive applications [33].

Since the requirements for CASs can vary diversely given different production scenarios, the design of them is frequently individualized. In addition to the selection of a suitable modality and the appropriate output device, factors such as the degree of support, adaptability to human needs (for example, the employee’s level of qualification), or the differing production environments compose roles that should not be neglected [34,35]. In summary, CASs may serve the paradigms of the efficient, networked factory as well as that of human-centered production, enabling flexible processes that can fundamentally meet product and process variations while ensuring high quality.

In this paper, we focus on stationary projection systems—more precisely light projection systems. For challenging work environments, such as those described in Section 1.1 using the example of aircraft production, the strengths of these systems come to fruition. Individual light projectors possess the ability to cover wide areas, as implemented in the event industry for large rooms, remaining relatively inexpensive to procure due to their availability on the consumer market. Due to this low cost, the systems can be deployed multiple times within a single workspace, mitigating potential occlusions caused by employees. Our goal is to provide a contribution to improve their applicability in CASs, especially for that of large products. In the following, we will look at the preliminary work related to commissioning, which in this case, mainly refers to the calibration process.

2.2. Commissioning of Visual Light Projectors

The utilization of moving head light projectors has been demonstrated previously as a part of the “Assist-By-X” assistance system [7]. The calibration process of the moving head is based on a precise calibration fixture with four reference points that define the workspace coordinate system. Based on the known dimensions of the fixture, the position and orientation can be calculated, resulting in a numerically stable solution that is expressed in the Denavit–Hardenberg notation [36], commonly used in robotics. The control of the moving head calibration process is aided by a Node-RED dashboard with a Python script in the back-end. The movement is controlled by MQTT through the Node-RED flows.

In an experiment prior to this paper, based on the proposed method, it became clear that the fixture itself must be positioned and aligned relative to a workpiece or reference coordinate system, which can pose further challenges and may require additional measurement equipment. This adds another step in the calibration process, including time and work effort, and is a potential source for uncertainty, motivating an approach to avoid this separate fixture.

2.3. Workpiece-Based Referencing

An approach for avoiding a separate calibration fixture by utilizing reference points on the workpiece itself for pose estimation and orientation correction has been demonstrated for aircraft wing assembly [37]. The specified goal was to increase the precision and speed of the wing-fuselage connection process in aircraft assembly. This was achieved by the continuous measuring of reference points on the aircraft wing with a laser tracker while its pose and position were manipulated by computer-controlled actors. The wing’s CAD model was used to find the workpiece coordinate system relative to the reference points. For the pose estimation, the Newton–Euler method, in combination with quaternions, was used.

Unlike the distinctly precise and immoderately priced laser trackers used in the paper, a system with low-cost moving head spots loses depth information, resulting in the light beam being represented by a vector from the moving head to the reference point with an unknown length. In order to use the remaining information, a different mathematical approach is necessary. This approach will be presented in the following chapters after an overview of the moving head control.

3. Fundamentals: Workpiece and Moving Head Kinematics

The large structure workpiece used for the development and verification of the calibration algorithm is a tail cone from aircraft production, and a schematic representation is shown in Figure 2. Typical in aircraft assembly, the component is constructed from metal sheets that are riveted together, leaving the heads of the rivets protruding relative to the surface. Rivets in characteristic spots, such as corners or intersections of rivet series, are chosen as reference points. These reference points are located on the lower half of the side of the workpiece, while the moving head is positioned on the ground, looking up at the reference points. The workpiece has a major diameter of about 2 m, a minor diameter of about 0.75 m and a length of about 3.5 m. It is held in a fixture, allowing for occlusion-free access to the aforementioned reference point area. On the basis of confidentiality, no pictures or renderings of the setup could be published in this paper.

The following section is divided into four subsections. The first subsection introduces the movement principle of the moving head and the relevant axis and parameters. This leads to the second subsection, that explains the control of the moving head with Cartesian coordinates. The third subsection introduces the coordinate systems and transformations used for the case of large structure workpieces. The last subsection gives an introduction to Newton’s method and the multivariable variant used for the iterative calculations in Section 4.

3.1. Axis and Control of the Moving Head

The moving head has two principal rotational movement axes called pan and tilt, which control the position of the light beam. Furthermore, it has zoom and focus parameters that control the size and sharpness of the light spot, as shown in Figure 2. These parameters are comparable to the intrinsic camera parameters within camera calibration, which will be further explained in the calibration section, Section 3. For effects, moving heads are typically equipped with a color filter wheel, one or more so called “Gobo wheels” with custom shapes, and a prism.

3.1.1. Pan and Tilt

The movement axes pan and tilt are the only axes that control the actual position of the light spot. For finer control, the finepan and finetilt controls exist, which are further explained in Appendix B.1. The movement ranges are typically in the area of $pan=\left[0^{\circ }{360}^{\circ }\right]/\left[0^{\circ }{540}^{\circ }\right]$ and $tilt=\left[0^{\circ }{180}^{\circ }\right]/\left[0^{\circ }{270}^{\circ }\right]$. The moving head is a serial kinematic, where $pan$ describes the rotation around the base, and $tilt$ describes the rotation of the actual projection head. This movement principle is similar to the definition of a spherical coordinate system and leads to the derivation of the Cartesian control in Section 3.2.

3.1.2. Focus

The moving head offers the possibility to adjust the light spot focus with its built-in optics. The correct focus setting depends on the distance between the moving head and the illuminated surface and is especially important when using differently shaped light spots. A properly focused spot reduces uncertainty regarding which point or area is intended to be highlighted by the assistance system and which is not. Overall, the aim of this assistance system is to provide a helpful guidance throughout a given workflow, which requires unambiguity. A detailed derivation of the relationship between the distance and the correct focus value for the moving head used in this paper can be found in Appendix B.2.

3.1.3. Gobo Wheels

The moving head used in this paper is equipped with two Gobo wheels, one of which being rotatable and the other having static elements. The static Gobo wheel is shown in Figure 3. In the original configuration, the Gobo wheels have artistic symbols and shapes for show usage, which only offer limited benefit for assistance applications. Consequently, a new Gobo wheel was designed and laser cut from sheet metal with smaller apertures to reduce the spot size. This measure increases the specificity of the spot when the moving head is supposed to illuminate an individual point. Other shapes, such as arrows, crosses, circles, or logos, are conceivable if one would like to enrich the illuminated position with additional information and/or instructions.

3.2. Control with Cartesian Coordinates

For interoperability of the moving head with other systems and equipment, it is necessary to control it with three-dimensional Cartesian coordinates instead of directly controlling the angles of each movement axis. The general approach follows the transformation equations from Cartesian coordinates to spherical coordinates. The equations for the $pa{n}^{\circ }$ and $til{t}^{\circ }$ angles are noted in Equations (1) and (2) and are generalized for point $P_n={[x_n,y_{n}m,z_n]}^T$, as shown in Figure 2 and Figure 4.

$$pa{n}_n^{\circ }=sgn\left({}^{MH}{y}_n\right)·arccos\left({}^{MH}\left(\frac{x_n}{\sqrt{x_n^2+y_n^2}}\right)\right)+{360}^{\circ }$$

(1)

$$til{t}_n^{\circ }=arcsin\left({}^{MH}\left(\frac{z_n}{\left|\right|{\overrightarrow{P}}_n\left|\right|}\right)\right)$$

(2)

Some changes and simplifications of the spherical coordinate transformation equations were made to adapt to the specific use case of a moving head: The $pa{n}^{\circ }$ equation in Equation (1) omits points with $z_n<0$ since the $tilt$ movement is software limited to the positive half space. Furthermore, the full 0–540${}^{\circ }$ movement range of the $pan$ axis is not used; instead, to avoid a full turn around of the moving head between certain orientations, the $pan$ axis is offset by 360${}^{\circ }$ to a range of 180–540${}^{\circ }$. The $til{t}^{\circ }$ Equation (2) is defined in a way that 0${}^{\circ }$ and 180${}^{\circ }$ are on the $xy$ plane, and 90${}^{\circ }$ is parallel to the z axis, which deviates from the typical definitions of spherical coordinates.

With the Cartesian control of the moving head established, it is important to define the notations and relations between the Cartesian coordinate system of the moving head and the world or reference coordinate system, from which the control inputs for the moving head will be provided.

3.3. Coordinate Systems and Transformations

As with every light projection system, a relationship between the moving head projector and the illuminated object, or more specifically, between the moving head coordinate system (${}^{MH}$) and the object coordinate system, called the reference coordinate system (${}^{Ref}$), has to be defined. This relationship, indicated by the red arrow in Figure 4, is expressed by the 4 × 4 homogeneous coordinate transformation matrix ${}_{Ref}{}^{MH}\mathbf{T}$ in Equation (3), which generally consists of a scale, zero to three rotations around the coordinate axis, and a translation vector. The scale is 1 in this case, but can be adjusted for coordinate systems with different-length units. The rotation part is expressed by the 3 × 3 rotation matrix ${}_{Ref}{}^{MH}{\mathbf{R}}_{zx}$, where the right side subscript indicates the order of the rotations performed around the designated axis. The translation is expressed by the translation vector ${[t_x,t_y,t_z]}^T$.

$${}_{Ref}{}^{MH}\mathbf{T}=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]=\left[\begin{array}{cccc}& & & t_x\\ & {}_{Ref}{}^{MH}{\mathbf{R}}_{zx}& & t_y\\ & & & t_z\\ 0& 0& 0& 1\end{array}\right]$$

(3)

The left side sub- and superscripts of the matrix symbols in Equation (3) follow the tensor notation, where the subscript designates the source and the superscript the destination coordinate system.

Calculating the transformation between the reference and the moving head coordinate system will be the core task of the calibration process explained in Section 4. Furthermore, it will be required for an assistance application in order to calculate the control inputs for the moving head from any given coordinate input in the reference coordinate system.

3.4. Usage of Non-Linear, Multidimensional Newton’s Method

The core of the calibration process from Section 4 are the previously derived Equations (1)–(3) from Section 3.2 and Section 3.3. These equations are similar in that they contain sine and cosine terms, making them non-linear equations, and the equations derived from them in the calibration process will be non-linear as well, ruling out the use of common linear solving algorithms. Although it would be possible to manually linearize the equations around the expected result, this process would require prior knowledge of the region that the result would lie in, which is often not possible. Furthermore, the solving algorithm must tolerate a certain degree of inaccuracy of the input values obtained from the manual reference point capturing, explained in Section 5.1.

A promising solving algorithm that fulfills the aforementioned constraints is the iterative Newton’s method, which performs automatic linearization at each iteration step. It can be adapted for multidimensional functions as well, and combined with its limited complexity, it was chosen for this paper. The general idea of Newton’s method is finding zeros of a function $f\left(x\right)$ (as defined in Equation (4)) where the conventional analytical approach is either not possible or infeasible [38].

$$f\left(x\right)\stackrel{!}{=}0$$

(4)

The first step in the iteration process is linearizing $f\left(x\right)$ at an initial $x^{\left(0\right)}$ value, which can be chosen arbitrarily or as an estimate close to the expected result, accelerating the convergence. In the second step, shown in Equation (5), the zero of this linearization is calculated, leading to the next $x^{\left(1\right)}$ value. The distance between the two values is called the iteration step size $\Delta x^{\left(0\right)}$.

$$x^{\left(1\right)}=x^{\left(0\right)}-\frac{f\left(x^{\left(0\right)}\right)}{f^{\prime }\left(x^{\left(x\right)}\right)}\stackrel{!}{=}{x}^{\left(0\right)}-\Delta x^{\left(0\right)}$$

(5)

This process is repeated until one of the three abortion criteria is met:

1.

The zero has been found with sufficient accuracy:

$f\left(x^{\left(k\right)}\right)\le {ϵ}_{f}with{ϵ}_f\approx 0$

This condition does not guarantee convergence but can be used if convergence is not a requirement.

2.

The difference between two x values fell below a specified threshold:

$\Delta x^{\left(k\right)}\le {ϵ}_{x}with{ϵ}_x\approx 0$

This condition signifies convergence but does not guarantee the zero has been found accurately.

3.

The maximum iteration step count K has been reached without fulfilling one of the other criteria. This usually means the iteration did not converge, or that it oscillates around the zero.

The one-dimensional approach can be extended to multiple variables and functions, utilizing a vector notation [39]. Since the algorithm solves for all variables at the same time, typically, a good trade off is reached, and the results require little to no further evaluation. The x value is extended to an $\overrightarrow{x}$ vector, which contains all iteration variables. As shown in Equation (6), the function Equation (4) is extended to a function vector $\overrightarrow{f}$ with the $\overrightarrow{x}$ vector as input.

$$\overrightarrow{f}\left(\overrightarrow{x}\right)=0$$

(6)

Equation (5) for calculating the next x value is extended. The derivations of $\overrightarrow{f}$ are combined in a Jacobian matrix $\overrightarrow{J}$, and a dampening factor $\alpha$ is introduced, leading to Equation (7). Depending on the function equations, unwanted oscillations around the result can occur. To dampen these oscillations and help with convergence, $\alpha$ limits the iteration step size. The damping is automatically increased as soon as the iteration step size no longer shows a significant change.

$${\overrightarrow{x}}^{(k+1)}={\overrightarrow{x}}^{\left(k\right)}-\alpha ·\Delta {\overrightarrow{x}}^{\left(k\right)}={\overrightarrow{x}}^{\left(k\right)}-\alpha {\left(\overrightarrow{J}\left(x^{\left(k\right)}\right)\right)}^{-1}·\overrightarrow{f}\left(x^{\left(k\right)}\right)$$

(7)

Since determining the inverse of $\overrightarrow{J}$ for each iteration step is numerically unfavorable, the linear Equation (7) is shuffled and solved for $\Delta {\overrightarrow{x}}^{\left(k\right)}$ instead, resulting in Equation (8).

$$\overrightarrow{J}\left({\overrightarrow{x}}^{\left(k\right)}\right)·\Delta {\overrightarrow{x}}^{(k+1)}=-\overrightarrow{f}\left({\overrightarrow{x}}^{\left(k\right)}\right)$$

(8)

As shown in Equation (9), for the multiple variable iteration, the second abortion criterion can be adapted by checking the norm of the iteration step size $\Delta \overrightarrow{x}$.

$$\left|\right|\Delta {\overrightarrow{x}}^{(k+1)}\left|\right|<{ϵ}_{x}with{ϵ}_x\approx 0$$

(9)

All necessary calculations within the iteration steps can be performed automatically and usually do not require manual tuning for different sets of input values. The value for $\alpha$ can be changed if necessary in the case that oscillations occur. With the outlined Newton’s method, it is now possible to derive the calibration process and define the function equations $\overrightarrow{f}\left(\overrightarrow{x}\right)$ from the core Equations (1)–(3). The next chapter will delve into further detail on this derivation.

4. Moving Head Calibration

The moving head alone is not (and can not) be aware of any other coordinate system other than its own, as it was not intended as a precision light spot projection system with Cartesian coordinate control. Additionally, it does not possess the necessary provisions to measure it’s position and orientation externally. As discussed in Section 2, the calculation of the moving head position and orientation relative to a reference coordinate system has been performed with a calibration frame by [7], and the utilization of known points on a large structure workpiece was conducted by [37]. The scope of the following sections involves the combination of these ideas as well as the calculation of the position and orientation of the moving head relative to the workpiece coordinate system, denominated as the reference coordinate system. Figure 5 gives an overview of the main steps in the calibration process as well as references to the sections describing each step.

The first phase of the calibration process is executed by illuminating distinctive points on the workpiece of which the exact coordinates in the reference coordinate system are known from the CAD construction data or, as shown later in the validation in Section 5.1, can be measured with external metrology equipment. The axis positions of the moving head are saved and used to calculate the moving head position and orientation relative to the reference coordinate system using an iterative process. The term ”Calibration” is not used in a sense to improve the accuracy of the system as it is known in measurement systems, but as an analogy to camera calibration in computer vision. The derived transformation is comparable to the extrinsic camera parameters of camera calibration, which determine the position and orientation of the camera in a reference coordinate system.

4.1. Determining the Moving Head Position

After illuminating the reference points and saving their distinct $pan$ and $tilt$ values in the first phase of the calibration process, the projector position is to be calculated in the second phase. The general steps of this phase are shown in Figure 6 as well as the input data and the result of the calculation.

The moving head’s light beam does not have a depth sensor; therefore, the distance between the moving head and the workpiece is not directly known. Since the rotation of the moving head relative to the reference coordinate system is also not known, the absolute rotation values of the $pan$ and $tilt$ axes are not directly usable to calculate the moving head position. Therefore, the reference points are divided into $\frac{N(N-1)}{2}$ duplicate free pairs, from which the angle differences $\Delta pan$ and $\Delta tilt$ between the two points of a pair can be calculated. These angle differences determine how far each axis has to move to get from one point to the other and no longer include the rotation of the moving head relative to the reference coordinate system. Furthermore, each point has a connecting vector in the reference coordinate system from the moving head to the reference point. The angle between these two vectors of each point pair can be decomposed into two angles that are equal to the $\Delta pan$ and $\Delta tilt$ angles from the moving head control. However, these vectors are not yet determined as the projector position is unknown, but they can be determined iteratively. The unknown projector position ${}^{Ref}{\overrightarrow{P}}_{MH}$ in the reference coordinate system will be used as the iteration variables in the $\overrightarrow{x}$ vector, as defined in Equation (10). The iteration variables can be considered as symbolic variables, so they can be used in the subsequent calculation without being known yet.

$${}^{Ref}{\overrightarrow{P}}_{MH}={}^{Ref}\left(\begin{array}{c}{x}_{MH}\\ y_{MH}\\ z_{MH}\end{array}\right)\equiv \overrightarrow{x}$$

(10)

The initial value ${\overrightarrow{x}}^{\left(0\right)}$ should be chosen close to the actual projector position since the Euler angles allow for more than one solution that the iteration algorithm can converge to. The minimum requirement for this estimation is the need for the initial position to be placed on the correct side of the work piece. A second, non-optimal solution exists, which is mirrored with respect to a plane formed by the reference points. Any measurement that offers results in the cm range is usually sufficient for a fixed or repeating setup. Previously calculated positions can be used as well. An overview of the points, angles and vectors for the following calculations is given in Figure 7.

The individual points in each point pair are index by n and m, which satisfies the condition $n,m\le N\wedge n<m$, with N representing the total number of points. The angle differences in each point pair are calculated as shown in Equations (11) and (12).

$$\Delta pa{n}_{n,m}^{\circ }=pa{n}_m^{\circ }-pa{n}_n^{\circ }$$

(11)

$$\Delta til{t}_{n,m}^{\circ }=til{t}_m^{\circ }-til{t}_n^{\circ }$$

(12)

The connecting vectors between each reference point and the moving head position are defined in Equation (13) with $i=n,m$. For readability, the index i will be used when an equation applies to n and m, and the ${}_{MH}$ subscript will be omitted.

$${}^{Ref}{\overrightarrow{v}}_{MH,i}={}^{Ref}({\overrightarrow{P}}_i-{\overrightarrow{P}}_{MH})={}^{Ref}\left(\begin{array}{c}{x}_i-x_{MH}\\ y_i-y_{MH}\\ z_i-z_{MH}\end{array}\right)\stackrel{!}{=}{}^{Ref}{\overrightarrow{v}}_i$$

(13)

Using the direct angle between ${}^{Ref}{\overrightarrow{v}}_n$ and ${}^{Ref}{\overrightarrow{v}}_m$ is not preferable in 3D space, as splitting it into two components equivalent to $pa{n}^{\circ }$ and $til{t}^{\circ }$ adequately functions. Therefore, if one component of the combined angle is much larger than the other, it conceals the other part. In order to reduce the complexity of this decomposition, the assumption is made that the ${}^{Ref}xy$ and ${}^{MH}xy$ planes are parallel or anti-parallel. This is due to the fact that rotations around three axes are indefinite, meaning that different solutions exist for the same change in orientation, precipitating unfavorable results and problems in the iteration. Since the workpiece is placed in a solid, alignable fixture, and the moving head can be easily aligned as well, the assumption can be considered valid. Therefore, both point pair vectors can be projected onto the ${}^{Ref}xy$ plane to obtain the ${}^{Ref}{\widehat{\overrightarrow{v}}}_n$ and ${}^{Ref}{\widehat{\overrightarrow{v}}}_m$ vectors in Equation (14), which are independent of $til{t}_i^{\circ }$ for $til{t}_i^{\circ }\in [0^{\circ },{90}^{\circ }[$.

$${}^{Ref}{\widehat{\overrightarrow{v}}}_i={}^{Ref}\left(\mathbf{P}·{\overrightarrow{v}}_i\right)={}^{Ref}\left(\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right)·\left(\begin{array}{c}{x}_i\\ y_i\\ z_i\end{array}\right)\right)={}^{Ref}\left(\begin{array}{c}{x}_i\\ y_i\\ 0\end{array}\right)$$

(14)

Therefore the angle between ${}^{Ref}{\widehat{\overrightarrow{v}}}_n$ and ${}^{Ref}{\widehat{\overrightarrow{v}}}_m$ can be equated to $\Delta pa{n}_{n,m}^{\circ }$ from Equation (11). Reshaping Equation (15) to a zero equation yields the first set of function equations (Equation (16)) for the iteration.

$$\Delta pa{n}_{n,m}^{\circ }\equiv arcccos\left({}^{Ref}\left(\frac{{\widehat{\overrightarrow{v}}}_n·{\widehat{\overrightarrow{v}}}_m}{\left|\right|{\widehat{\overrightarrow{v}}}_n\left|\right|·\left|\right|{\widehat{\overrightarrow{v}}}_m\left|\right|}\right)\right)$$

(15)

$$f_I:0=arcccos\left({}^{Ref}\left(\frac{{\widehat{\overrightarrow{v}}}_n·{\widehat{\overrightarrow{v}}}_m}{\left|\right|{\widehat{\overrightarrow{v}}}_n\left|\right|·\left|\right|{\widehat{\overrightarrow{v}}}_m\left|\right|}\right)\right)-\Delta pa{n}_{n,m}^{\circ }$$

(16)

For deriving the $\Delta til{t}_{n,m}^{\circ }$ angle, both vectors must be on a plane orthogonal to the ${}^{Ref}xy$ plane, which makes them independent of the $\Delta pa{n}_{n,m}^{\circ }$ angle. This condition can be satisfied by rotating the ${}^{Ref}{\overrightarrow{v}}_m$ vector around the z axis by $-\Delta pa{n}_{n,m}^{\circ }$, as shown in Figure 7 and Equation (17). In conjunction with the resulting vector ${{}^{Ref}\widehat{\overrightarrow{v}}}_m$, the accompanying point ${}^{Ref}{\overrightarrow{P}}_m$ is rotated as well. Contrary to the simplified figure, the rotated point ${{}^{Ref}\widehat{\overrightarrow{P}}}_m$ does not lie on the workpiece surface anymore, as the length of the connecting vector did not change $(\parallel {\widehat{\overrightarrow{v}}}_m\parallel =\parallel {\overrightarrow{v}}_m\parallel )$. This does not have an impact on the further calculations, as only the angle between the vectors are relevant, and the vectors are normalized in Equation (18).

$${}^{Ref}{\widehat{\overrightarrow{v}}}_m\stackrel{!}{=}{\mathbf{R}}_z{(-\Delta pa{n}_{n,m}^{\circ })}^T·{}^{Ref}{\overrightarrow{v}}_m$$

(17)

Analog to Equation (15) for $\Delta pa{n}_{n,m}^{\circ }$, Equation (18) is derived for $\Delta til{t}_{n,m}^{\circ }$. The second set of functions in Equation (19) for the iteration is then equivalent to Equation (16), now with the rotated second vector instead of the projected vector.

$$\Delta til{t}_{n,m}^{\circ }\equiv arcccos\left({}^{Ref}\left(\frac{{\overrightarrow{v}}_n·{\widehat{\overrightarrow{v}}}_m}{\left|\right|{\overrightarrow{v}}_n\left|\right|·\left|\right|{\widehat{\overrightarrow{v}}}_m\left|\right|}\right)\right)$$

(18)

$$f_{II}:0=arcccos\left({}^{Ref}\left(\frac{{\overrightarrow{v}}_n·{\widehat{\overrightarrow{v}}}_m}{\left|\right|{\overrightarrow{v}}_n\left|\right|·\left|\right|{\widehat{\overrightarrow{v}}}_m\left|\right|}\right)\right)-\Delta til{t}_{n,m}^{\circ }$$

(19)

With the two sets of function equations, Equations (16) and (19), the moving head position, which was defined and initialized in Equation (10), can be calculated iteratively.

4.2. Determining the Moving Head Orientation

The third and final phase of the calibration is the calculation of the moving head orientation relative to the reference coordinate system. The steps of this phase are shown in Figure 8.

To complete the final transformation matrix between the reference and moving head coordinate system introduced in Section 3.3 (Equation (3)), the 3 × 3 rotation matrix ${}_{Ref}{}^{MH}\mathbf{R}$ must be calculated. Since the angles of the rotations are yet to be determined, it is constructed with the symbolic variables $\alpha$ for the rotation around the x axis and $\gamma$ for the rotation around the z axis. The rotation around the y axis has been omitted in this case as it is assumed again that the ${}^{MH}xy$ and ${}^{Ref}xy$ planes are parallel or anti-parallel, which leads to the transformation matrix shown in Equation (20). The values of $\alpha$ are limited to $\alpha =\{0^{\circ },{180}^{\circ }\}$. The angle $\gamma$, however, can take any value in the range of $\gamma =[-{180}^{\circ },{180}^{\circ }[$. The $\overrightarrow{x}$ vector for the iteration, therefore, is ${\overrightarrow{x}}^{\left(0\right)}={[\alpha ,\gamma ]}^T$. Estimating initial values for the ${\overrightarrow{x}}^{\left(0\right)}$ is trivial for $\alpha$ in this case due to the discrete values. For $\gamma$, a qualitative estimate is usually sufficient.

$${}_{Ref}{}^{MH}{\mathbf{R}}_{zx}={\mathbf{R}}_z{\left(\gamma \right)}^T·{\mathbf{R}}_x{\left(\alpha \right)}^T={\left[\begin{array}{ccc}cos\gamma & sin\gamma & 0\\ -sin\gamma & cos\gamma & 0\\ 0& 0& 1\end{array}\right]}^T·{\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\alpha & sin\alpha \\ 0& -sin\alpha & cos\alpha \end{array}\right]}^T$$

(20)

With this symbolic rotation matrix, Equation (21) for the translation vector ${[t_x,t_y,t_z]}^T$ of the transformation matrix can be determined from the moving head position calculated in Section 4.1.

$$\left(\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right)\stackrel{!}{=}-{}_{Ref}{}^{MH}{\mathbf{R}}_{zx}·{}^{Ref}{\overrightarrow{P}}_{MH}$$

(21)

With the rotation matrix and the translation vector, the transformation matrix ${}_{Ref}{}^{MH}\mathbf{T}$ is completely defined and can be used to transform the reference points from the reference coordinate system ${}^{Ref}$ to the moving head coordinate system ${}^{MH}$, according to Figure 4. The transformation step in Equation (22) yields the ${[x_i,y_i,z_i]}^T$ coordinates for the next calculation step.

$${}^{MH}{\overrightarrow{P}}_i={}_{Ref}{}^{MH}\mathbf{T}·{}^{Ref}{\overrightarrow{P}}_i$$

(22)

The transformed reference points can now be used with the $pa{n}^{\circ }$ and $til{t}^{\circ }$ values from the reference points in combination with Equations (1) and (2) to derive the second set of function Equations (23) and (24) for the next iteration. This results in $N(N-1)$ equations for N points.

$$\begin{array}{cc}\hfill f_{III}:& \hfill 0=sgn({}^{MH}{y}_i)·arccos\left({}^{MH}\left(\frac{x_i}{\sqrt{x_i^2+y_i^2}}\right)\right)-pa{n}_i^{\circ }\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill f_{IV}:& \hfill 0=arcsin\left({}^{MH}\left(\frac{x_i}{\left|\right|{\overrightarrow{P}}_i\left|\right|}\right)\right)-til{t}_i^{\circ }\hfill \end{array}$$

(24)

With the second set of equations and the initial estimate of the orientation angles ${\overrightarrow{x}}^{\left(0\right)}$, the moving head orientation relative to the reference coordinate system origin can be calculated, leading to the final transformation between the moving head and the reference coordinate system, thus completing the calibration process. The final transformation matrix can be found in Equation (A6). In order to verify the exactitude and applicability of the obtained transformation, and the system in general, the following section will discuss the validation of everything shown thus far.

5. Validation

After establishing the control and calibration of the moving head in the previous sections, Section 3 and Section 4, the performance of the overall system is to be evaluated. This section has two main parts: First, the possible error sources introduced by the reference point capturing, control and mechanical limits of the moving head and the algorithmic accuracy are discussed in the first three sections, Section 5.1, Section 5.2 and Section 5.3. Since each of these steps has an influence on the following step, it is important to assess each possible error contribution to the overall error and evaluate their respective significance. Second, the practical validation process is described and evaluated in Section 5.4. The setup for this is equal to the setup of the calibration process described in Section 3, with the moving head located on the ground pointing at the reference points on the tail cone, as illustrated in Figure 9. The theoretical limits can then be set in relation to real world accuracy.

5.1. Capturing Reference Points

For the workpiece used in this paper, no CAD data were available to extract the reference point coordinates from. For this reason, a Leica LTD800 Laser Tracker was used, which offers an absolute measurement precision of less than 50 $\mathsf{\mu }$m in the range that was used in this paper [40]. This precision far exceeds the expected accuracy of the moving head and therefore will be assumed to be sufficient as a reference. This approach can likewise be applied if CAD data are available; however, for example, the workpiece is still flexible before final assembly, which is typical for large, flat workpieces such as side panels that do not possess much structural integrity independently.

The reference points chosen were rivets in distinctive positions, such as on an intersection of a horizontal and vertical rivet series. As described in Section 3, the rivet heads stand proud of the surface, rendering them easy to distinguish from the surface itself. Since the light spot, even with the smallest aperture, is much larger than a rivet head, a target printed on paper was placed over the rivets to aid with centering the light spot, similar to Figure 10. The $pan$, $finepan$, $tilt$ and $finetilt$ angles of each reference point were saved and used for the calibration process (see Appendix A.2).

It should be noted here that this procedure offers only limited accuracy since the light spot on the curved surface of the workpiece deviates distinctly from a perfect circle in most instances. On a flat surface that is not perpendicular to the light beam, the light spot would become a non-symmetric ellipse; however, on the curved surface, the resulting shape is no longer easily parametrizable. The effect is shown in Figure 9. Therefore, the $pa{n}^{\circ }$ and $til{t}^{\circ }$ of the reference points should only be regarded as a best-effort value, with an uncertainty in the mm range.

5.2. Theoretical and Mechanical Limits

From the derivation of the $pa{n}^{\circ }$ and $til{t}^{\circ }$ axis control in Appendix B.1, the limits of the control input accuracy can be derived. For the moving head used in this paper, it is $pa{n}_{step}=0.008272\frac{{}^{\circ }}{dig}$ and $til{t}_{step}=0.002747\frac{{}^{\circ }}{dig}$. The testing showed that the moving head still performs movement for single-digit ($dig$) increments and decrements; however, the resulting accuracy of the small movements was not further evaluated.

Comparable to industrial robot systems, the moving head can be characterized in terms of pose accuracy and pose repeatability [41]. The Pose Accuracy determines how precise an arbitrary given point will be hit by the light spot. The Pose Repeatability determines how precise the same point will be hit from different starting points and different movement velocities between the start and end point. The Pose Accuracy is generally harder to determine for light projection systems as the light beam does not have an end point until it contacts a surface. The pose repeatability, however, can be assessed without complication for isolated cases. From those results, a general idea about the systems accuracy can be derived.

To gain an initial estimation of the pose repeatability, a simple experiment was conceptualized as shown in Figure 11. A target, as shown in Figure 10, is aligned to an arbitrary center point. The moving head is moved to a point on a concentric circle around this center point and then back to the center. The deviation from the actual center point is measured and saved. Since the movement speed depends on the distance, and since the movement path chosen by the moving head is not always the shortest or direct path, different circle diameters and positions on the circle must be tested. It is posited that the initial orientation of the axis, especially the $tilt$ axis, has a significant influence on the pose repeatability due to the backlash within the moving head mechanism.

The described pose repeatability experiment was carried out by mounting the target shown in Figure 10 onto the workpiece. This way, realistic and relevant measurements can be generated, as the actual distance and orientation are used. A variety of starting points was selected, from far apart (>1 m) to achieve a high movement velocity, to short distances (5 mm) to achieve a very slow movement velocity. The starting points included single-axis movements and combined movements. However, none of the starting points could offer a measurable deviation from the marked center point to be observed. The achieved pose repeatability of the described experiment exceeded the expectations.

When manually agitating the moving head to forcefully alter its $pan$ and $tilt$ angles, it moves back precisely to the original position, indicating that the axes are servo controlled or have position feedback, which is a possible explanation for the high level of accuracy observed.

In order to gain more than a qualitative understanding of the pose repeatability, a laser tracker was used to directly measure the position at the light exit window. A retro reflector was mounted on the moving head, and the same experiment as described previously was conducted. After each return to the center of the target, the position of the retro reflector was measured with the laser tracker. The measurements and results are listed in

Table A3. Measurements of the moving head pose repeatability. The coordinates exist in the laser tracker coordinate system. ${\sigma }_{LT}$ is the measurement uncertainty reported by the laser tracker.

Index	X in mm	Y in mm	Z in mm	${\mathit{\sigma }}_{\mathbf{LT}}$ in mm
1	−306.4154	−2032.1464	−972.1941	0.00188634
2	−306.4316	−2032.1124	−972.2038	0.00220966
3	−306.2175	−2032.1691	−971.9397	0.00263319
4	−306.2847	−2032.1653	−971.9680	0.00199971
5	−306.2746	−2032.1472	−971.9647	0.00221153
6	−306.2349	−2032.1175	−971.9473	0.00211342
7	−306.2486	−2032.1570	−971.9434	0.00228444
8	−306.2201	−2032.1406	−971.9422	0.00236737
$\mu$	−306.290925	−2032.144438	−972.0129	0.002213208
$\sigma$	0.079785928	0.019298765	0.107880628	0.000214366

, and the resulting standard deviations are ${\sigma }_x$ = 79.79 $\mathsf{\mu }$m, ${\sigma }_y$ = 19.30 $\mathsf{\mu }$m and ${\sigma }_z$ = 107.88 $\mathsf{\mu }$m, while the reported standard deviation from the laser tracker for its position measurements is a magnitude lower, indicating that the measurements are sufficiently precise. While these values do not give information regarding the pose repeatability of the light spot, they can be used to determine whether or not the accuracy is sufficient for the application.

Altogether, the pose repeatability has a much smaller influence on the overall system accuracy than the spot distortion discussed previously.

5.3. Algorithmic Accuracy

With the high mechanical accuracy determined in the pose repeatability experiment, the focus will now be set on evaluating potential error magnitudes and sources resulting from the control input side of the projection system. Iterative algorithms in combination with measurement errors are not able to deliver a perfect solution, but they can satisfy the given metrics as adequately as possible (if possible, and if convergence was reached). The metrics of the presented algorithm are Equations (16), (19), (23) and (24). As described previously, these equations should be as close as possible to 0, and all of these equations express angles. The errors are assumed to be small, enabling the neglect of the non-linear characteristics of the errors and the observing of the mean values for possible systematic errors and the standard deviation for potential outliers. Since the mean value and standard deviation of the position estimation in Equations (16) and (19) originates from the angle difference between two vectors, it is impossible to directly correlate them to the accuracy of the position estimation. However, it is possible to assess the quality of the result in terms of moving head steps. Ideally, the error should be smaller than one step, as this would limit the algorithmic accuracy to below the mechanical limits.

For the analysis of the transformation estimation error, it is assumed that the position error is small and that the light beam contacts a flat surface perpendicularly, eliminating spot distortion. Considering that the exact surface will be neglected, and no distance information can be derived from the angle error, it is not possible to reconstruct meaningful Cartesian coordinates. Therefore, it is expedient to combine the angle errors of the $pa{n}^{\circ }$ and $til{t}^{\circ }$ angles of each point to a single angle error, as shown in Equation (25).

$$\Delta combine{d}^{\circ }=arccos(cos(\Delta pa{n}^{\circ })\ast cos(\Delta til{t}^{\circ }))$$

(25)

With this combined angle error $\Delta combine{d}^{\circ }$, the point position error e can be calculated (Equation (26)) as a circle around the true center of a point $P_n$, as seen in Figure 12. The distance d is calculated from the estimated projector position and the Cartesian coordinates of the reference point $P_n$. From the individual point position errors, a mean value and standard deviation can be calculated to evaluate the quality of the final result.

$$e=d·sin(\Delta combine{d}^{\circ })$$

(26)

The mean values, standard deviations, and equivalent step counts are listed in

Table 1. Results from the algorithmic accuracy analysis of the position and transformation estimation and the combined point position error.

Equation	Angle	$\mathit{\mu }$	$\mathit{\sigma }$	${\mathit{\mu }}_{\mathit{steps}}$	${\mathit{\sigma }}_{\mathit{steps}}$
(16): $f_I$	$\Delta pan$	−0.01492${}^{\circ }$	0.1557${}^{\circ }$	1.80	18.82
(19): $f_{II}$	$\Delta tilt$	0.01748${}^{\circ }$	0.2026${}^{\circ }$	6.36	73.75
(23): $f_{III}$	pan	0.05552${}^{\circ }$	0.1056${}^{\circ }$	6.71	12.77
(24): $f_{IV}$	tilt	−1.0343${}^{\circ }$	0.1406${}^{\circ }$	−376.52	51.18
(26): e	Combined	0.02640 m	0.00482 m	-	-

for the position and transformation estimation as well as the point position error. The values of the position estimation suggest an adequate result, with the $pan$ angle being more accurate than the $tilt$ angle. After the transformation estimation, a large error in the mean value of the $tilt$ angle can be observed, while the $pan$ angle error remains in the same magnitude. This deviation propagates to the point position error, where the mean value is much larger than that of the standard deviation.

The low error at the position estimation stage suggests that a result with an adequate fit for the given input values was found, which, however, deviates from the true projector position and therefore led to the mean value deviation in the $tilt$ angle. After manually tuning the result of the estimated moving head position, it was found that the z component was the cause of the inaccuracy. A hypothesis for the source of this inaccuracy is the much wider $pan$ angle range of the reference points compared to the $tilt$ angle range (see

Table A2. Reference points in moving head DMX Channel values, measured coordinates, and lower and upper position error between jogged and calculated reference points.

Index	pan	finepan	tilt	finetilt	X in m	Y in m	Z in m	${\mathit{e}}_{\mathit{l}}$ in mm	${\mathit{e}}_{\mathit{u}}$ in mm
1	192	202	45	84	−1.7443	0.2477	−0.3072	5	5
2	185	246	55	235	−1.6257	0.5788	−0.2365	3	6
3	173	29	62	0	−1.5049	1.0239	−0.1801	3	4
4	160	209	65	190	−1.3521	1.4272	−0.0178	3	3
5	150	226	58	10	−1.2126	1.9556	0.1109	4	6
6	153	73	50	73	−1.3299	1.9238	−0.0744	3	6
7	160	36	52	166	−1.4477	1.5623	−0.1810	6	6
8	172	22	50	241	−1.6119	1.1009	−0.3073	5	6
9	180	236	44	190	−1.7302	0.7689	−0.3858	4	4
10	189	52	35	0	−1.8702	0.3832	−0.4739	5	5
11	160	170	45	225	−1.5453	1.6221	−0.2528	5	5
$\mu$								4.18	5.09
								4.64
$\sigma$								1.03	1.00
								0.83

), since the x and y components are accurate and have a stronger correlation to the $pan$ angle. This hypothesis is supported by the fact that the accuracy of the result increases when selecting reference points with an overall smaller $pan$ range, even though the overall number of points is decreased. Since this was the result of a qualitative, manual tuning process, the results are not described in detail on account of their limited generality. Overall, special consideration should be taken when selecting reference points to avoid the adverse effects of data bias. The effect of oscillations around a result was reduced with the dampening factor $\alpha$ introduced in Equation (7) in Section 3.4. Convergence to local minima can be avoided by selecting the starting values of the iteration in the same general area as the expected result.

5.4. Practical Validation

The point position errors calculated previously offer only a qualitative significance for the given tail cone due to the simplifying assumptions regarding the surface being made, which do not correlate well to the particularly curved surface of the actual part, thus motivating a real-world validation. For the validation, the reference points are overlaid with a target similar to Figure 11, and the outlines of the light spots of the manually jogged position and the calculated position are manually traced.

This manual process is necessary due to the aforementioned distortion of the light spot, and on the curved surface of the workpiece especially, the center point is difficult to reconstruct. As the spot size of the two ellipses is identical, there will always be two intersection points and two points with maximum distances, as shown in Figure 13. Due to the angled surface, one spot will be elongated more than the other one, resulting in two distances, named $e_l$ for the lower and $e_u$ for the upper distance (comparable to the e value from the algorithmic accuracy evaluation). The measurements are listed in Table A2, along with the mean values and standard deviations.

The combined mean value is 4.64 mm, and the combined standard deviation is 0.83 mm. These values are significantly lower than the predicted theoretical values, implying that the simplifications made in Section 5.3 may not be applicable to all cases. Particularly, the angle between the light beam and the surface in the real world experiment is much shallower than the perpendicular angle assumed in the simplification. Depending on the general orientation between moving head and workpiece, the real-world errors may become more or less favorable compared to the theoretical values.

6. Discussion

After completing the validation of each aspect in the calibration process, this section evaluates the individual results and puts them into relation with each other. The order of such follows the order of the previous Section 5. The first paragraph covers the theoretical validation of the reference points and the moving head. The second paragraph focuses on the calibration algorithm and the aspect of using Newton’s method. The third and final paragraph discusses the results of the practical validation.

Capturing the reference points with a laser tracker adequately functioned in this case, delivering valid and usable results. This method can be applied to most large-structure workpieces or work areas if no CAD data is available, or if the expected data accuracy is insufficient.

The described point distortion posed significant issues and was a limiting factor for the accuracy when jogging the light spot to the reference points. An unclear but potentially significant amount of uncertainty was rendered at this stage, owing to the difficulty in determining the exact spot center. In all probability, this outweighed the other error sources discussed.

In the validation of the movement control, it was shown that the step granularity of the control input allows for precise movement inputs. Similarly, the results of the qualitative and laser-tracker-based pose repeatability experiment indicated that the moving head used in this paper is capable of achieving a precise mechanical accuracy in practice as well. It should be noted, however, that mechanical and control accuracy vary for different devices and application scenarios. For example, at greater distances between the moving head and the workpiece, small angular errors have a larger effect.

Obtaining meaningful measures for the algorithmic accuracy proved challenging due to both the nature of iterative algorithms and the non-trivial surface of the respective workpiece. The proposed point position error metric only offered limited relevance and predicted a low overall accuracy. Furthermore, a discrepancy between the two steps of the algorithm were discovered, where a suboptimal result of the first step can perpetuate a high error in the second step. Currently, no feedback from the second to the first algorithm step is in place to counteract such effects. The distribution of the reference points embodies a direct influence on the accuracy of the result components if certain input characteristics outweigh others.

None of these aforementioned effects can be attributed to the use of Newton’s method for the calibration; however, in the current implementation, the control authority over the results is limited. While Newton’s method converged to a result reliably, it was not necessarily the correct or desired result, attributable to multiple local minima that can occur in the multidimensional function equations. The calculation speed of the calibration algorithm increases significantly with an increased point count N since the number of function equations scales with $O\left(N^2\right)$, leading to larger Jacobian matrices and more substitution steps. Further information regarding relevant performance metrics can be found in Appendix C.3. While improved solving algorithms exist that are less susceptible to local minima and offer higher performance, they embody increased complexity for the implementation. Overall, Newton’s method offered an adequate compromise between performance and complexity and was sufficient for the proof-of-concept.

The practical validation was able to demonstrate a higher-than-predicted point position accuracy, revealing a discrepancy between the algorithmic accuracy analysis and the real-world example. It was shown that the proposed method is applicable to large-structure workpieces with non-trivial surfaces (as intended) with reasonable accuracy. However, the exact accuracy is difficult to quantify in view of the resulting manual processes and judgements regarding the aforementioned spot distortion, which potentially introduces further unidentified uncertainty.

7. Summary

Throughout the course of this paper, the relevance and applicability of a moving head as a Cognitive Assistance System for large-structure workpieces was established. The control of the moving head was explained, and methods for the characterization and modification of the device were presented, improving the usability as an assistance system. A detailed decomposition of mathematical relations between the moving head and the reference coordinate systems were derived, giving a generalized foundation for further applications. This includes moving heads and similar light projection systems based on angular control inputs. The core of this paper is the novel, iterative calibration algorithm based on Newton’s method, enabling a workpiece-based calibration without additional external reference points or fixtures. In the literature on light projection systems, the presented calibration approach is currently unique and superior to the only previously presented calibration method based on four precisely placed reference points on a calibration frame by [7]. It can be suspected that laser projection systems use similar calibration algorithms; however, no public literature or documentation about the internal functionality of those algorithms has been released. Therefore, a direct comparison to the proposed algorithm is not possible. The validation uncovered the various challenges and constraints of the system and proposed algorithm, identifying the point distortion as a significant factor of measurement uncertainty. In summation, a reasonable accuracy could be proven, validating the moving head spot as a useful, low-cost assistance system with control via MQTT for ease of integration into an industrial environment, as established previously by [7]. Due to the low unit price, occlusion problems may be addressed by scaling the assistance system with multiple moving head spots, mitigating information loss.

8. Outlook and Future Work

This paper presented a comprehensive range of future research prospects, which can be broadly categorized into three distinct clusters. The first over-arching cluster involves the evaluation of cognitive assistance applications with a moving head; a special focus can be applied to its existing workflow integration as well as further research regarding worker acceptance. Another focus within this cluster can be applied to the scaling of the assistance system with multiple moving heads, sharing the dynamics of a common workspace as well as other assistance systems in an effort to avoid occlusion by objects, tools and workers. Different shapes for the Gobo wheels can be explored to enrich the light spot with simple information.

The second cluster entails improvements to the iterative algorithm. Several approaches are possible, including the use of other non-linear solvers for the derived equations, or the merging of two iteration steps to solve the aforementioned discrepancy between the steps. While Newton’s method is simple to implement, it requires a reasonably good estimation of the result to avoid local minima. A logical next step is therefore evaluating different iteration algorithms for improved performance and robustness. Weighing factors could also be introduced to tune the results in the case of the heterogeneous distribution of angle ranges on the reference points as well as potential methods to reduce the amount of reference points if they offer little to no additional accuracy for the end result, thus simultaneously increasing performance. Manual plausibility checks in the case of multiple solutions due to Euler’s angles could also be introduced. To mitigate this problem, especially within applications where assumptions made about the coordinate system do not hold, the use of quarternions instead of Euler’s angles can be examined. An additional focus can rest on quality measures of the calculated result in order to improve the validity assessment, and potentially, the feedback of metrics into the algorithm.

The third and final cluster expresses the characterization of the moving head itself. One aspect of this characterization entreats further research into the mechanical accuracy of each mechanical and optical axis with the methods described in this paper. Using a laser tracker, it is possible to determine pose accuracy and repeatability. With a mathematical description of the mechanical zoom, in conjunction with the focus and either custom Gobo wheels or a separate iris, a variable and controllable spot size at the workpiece is achievable. The custom Gobo wheels can be used for a precompensation of the aforementioned spot distortion, which leads to vast uncertainty. For this precompensation, a mathematical description of this distortion in combination with different workpiece surfaces is necessary.

The presented features and capabilities of a moving head spot, in conjunction with further research, situates the moving head as a viable low-cost alternative to established light projection systems, incentivizing its deployment in the production environments of large scale products.