Presented at the XII International Conference on the Use of Computers in
Radiation Therapy (XII ICCR), May 27-30, 1997, Salt Lake City, UT.
Refer to the presentation for illustrations
of some of these principles.
3D visualization techniques are commonly employed in radiation treatment
planning (RTP). The resulting pictures are often pretty, and a tendency
exists to view these as inherently correct, since they look so much like
the reality we think they accurately represent. However, it can be very
difficult to know what is really being displayed and whether it is being
perceived and interpreted accurately by the viewer, as the visualization
community is well-aware. The visualization process can suppress information
about the original data quality, add information not present in the
original data, and lose information. Different visualizations of the same
data can appear to disagree. Tradeoffs are often made between speed and
accuracy, possibly without understanding the errors. All of this can result
in incorrect assumptions about the structure or content of the original
data or the meaning of the visualization, but the observer is often unaware
of inaccuracies. It is important to understand the significance of these
issues in the overall RTP process. Whether any of them really matter
depends on the goal of the visualization and the difference between the
actual data and what is being conveyed to the observer. If this error could
be measured, more meaningful visualizations could be created, leading to
more accurate and consistent interpretations of the data. Tradeoffs could
be assessed during RTP system development, different systems could be
compared, and optimal visualization techniques could be investigated. The
goal of this paper is to initiate discussion on this problem and on
possible solutions. First, some common RTP visualization issues are
analyzed in an initial effort to characterize the problem. Then, one
possible method of modeling and evaluating these issues is presented as a
strawman proposal. Only the surface of these issues is scratched, but
hopefully a framework is provided for continued investigation.
General Visualization Issues
"Visualization is the process of mapping numerical values onto
perceptual dimensions" . Ideally, our
interpretation of the result should effectively and accurately represent
the original data. Two primary issues can obscure this goal:
- the process used in creating the visualization, and
- the process by which the display is perceived and
Visualization can suppress information about original data quality, for
example, by interpolating or otherwise representing missing data . Visualization can add information not present
in the original data. Mapping dose data to a rainbow color map gives the
erroneous impression of values clustered in discrete bands . Sometimes, more of the information bandwidth
in the visualization comes from the modeling than from the actual data , such as smooth shading of a polygonal isodose
surface, which gives a false impression of the granularity of the original
dose matrix. Even a very coarse dose matrix can generate rather smooth
isosurfaces. Visualization can hide information. Mapping dose data to a
rainbow color map can hide significant detail within each color band. One
might argue that polygonal isodose surfaces should be displayed with flat
shading to accurately portray the dose matrix granularity. However, if an
algorithm other than marching cubes is used or the surface has been
simplified, the granularity might now appear more coarse than it really is.
Different visualizations of the same data can disagree. A dose volume
histogram (DVH) based on dose voxels might disagree with the display of the
structure surface with dose mapped to it or with a dose isosurface on top,
since the latter dose mappings are likely based on dose points.
In any computation, a tradeoff can exist between performance and
accuracy. To balance this tradeoff, one often simply assesses the error
between the correct answer and the computational result. However, with
visualization, both the correct answer and the computational result can be
difficult to determine. The computational result is our interpretation of
the visualization, while the correct result is what our interpretation of
the raw data would be without the visualization. This is clearly not a
simple calculation. Tradeoffs are nevertheless being made in RTP systems,
possibly without an understanding of the errors and almost certainly
without communication of these issues to the users.
Further, we are prone to have a false sense of security due to the
nature of RTP visualizations. These generally attempt to parallel the
physical meaning of the data. Examples include dose volume information
displayed as a 3D isodose surface, creation of a structure surface from a
collection of CT contours, or an arbitrary slice through a CT volume. Such
visualizations, which appear to be direct mappings from the original data
space, can lead one to view them as inherently "correct" . This is in contrast to less intuitive
techniques, such as using parallel coordinates to display multivariate
These issues are not specific to RTP. The young field of scientific
visualization still wrestles with them [5 , 6 , 7 , 1]. Little has been done to define a good
visualization or visualization system; to provide guidance in making
reliable, accurate visualizations; or to provide benchmarks for designing,
validating, or comparing visualization products . However, our task should be much more tractable
since it is very well-bounded, with well-understood goals.
It is important to understand the significance of these issues in the
overall RTP process. Whether any of them really matter depends on the goal
of the visualization and, in the end, the difference between what is
desired to be conveyed and what is actually conveyed to the observer.
Essentially, we are looking for an error measurement, albeit a complex one!
The first step must be to understand the problem clearly. Then we can look
for ways to measure and communicate this error.
RTP Visualization Issues
Some common RTP visualization issues are now discussed to begin to
characterize the problem. In each case, consider how these issues can be
characterized, evaluated, quantified, and communicated, such that one can
understand the possible error in the results and compare results from
Isosurfaces and Surface Simplification
In the commonly used marching cubes algorithm , calculation time depends on the size of the
data set, while the complexity of the surface depends on the voxel size.
Thus, even a very smooth (low spatial frequency) surface can consist of a
large number of triangles, resulting in reduced interactive performance.
Both problems can be simply addressed by first spatially filtering the
volume data, mapping to a lower resolution 3D grid. Calculation time will
decrease and surface complexity will decrease, yielding increased
interactivity, but at the cost of information loss. For dose information,
this can be dangerous. Note, the resulting isosurface will likely appear
even smoother than the original.
Newer, more efficient approaches to isosurface generation can reduce
calculation time and resulting surface complexity (e.g., ). However, these are research topics and might
not be available in the chosen visualization package, or limited
development resources might preclude these as options.
One safe approach is to accept the calculation time, then apply a
surface simplification algorithm to reduce the number of triangles in the
mesh (improving interactivity) while maintaining a "good"
approximation to the original geometry. How good? There have been many
approaches to this problem, with different error measurements and
tolerances. In a recent powerful approach from IBM , the original volume is fully preserved and a
maximal surface error is rigidly enforced at each vertex, while favoring
equilateral triangles. However, this can take longer to run than less
Thus, the following tradeoffs exist.
- Use the original data, and display as it is. Surface
generation time and interactivity may be too slow, but
information is not lost.
- Spatially filter the volume data up front. This speeds up
both surface generation and interactivity, but
information is lost. The information lost is proportional
to the speed-up.
- Use the original data, followed by surface
simplification. Surface generation is slower than (1),
but interactivity speeds up. Accuracy depends on limits
of the surface simplification algorithm.
- Use the original data, followed by "accurate"
surface simplification. Surface generation time is even
slower than (3), but with similar interactivity speeds;
accuracy, however, will be much better defined.
When observing a given isosurface, how can one know which tradeoffs were
made, and what is the possible error? When visualizing dose, these issues
can be critical.
Surfaces are usually displayed with smooth shading. It not only looks
prettier than flat shading, but the latter can cause reduced interactivity.
Smooth shading both adds and hides information. For example, with marching
cubes, the surface triangle size directly indicates the data voxel size.
Displaying these triangles with flat faces clearly shows the original
volume data granularity. Smooth shading adds information by interpolating
values at the display pixel resolution, thus hiding the volume data
granularity. Of what value is this added information?
Consider two smoothly shaded isosurfaces, one calculated from the
original volume data, the other from grossly reduced volume data. Both
would appear smooth. Flat shading would clearly show the difference.
However, if the surface has been simplified, the original voxel information
is lost anyway, and flat shading would incorrectly portray the original
dose matrix as more coarse than it was. When looking at a surface, it can
be difficult to assess the data used in its calculation.
Surface Generation from Contours
Two common approaches are employed to generate a surface from a set of
contours. One is to connect contours between adjacent planes (e.g., ). Accuracy issues can arise from the
connection algorithm (e.g., how it handles structures that branch or that
come and go) and from contour or surface simplification methods. The second
approach is to render (fill) the contours, stack these images to form a
volume, then create an isosurface. Accuracy issues can arise from the
rendering resolution, the isosurface algorithm, and surface simplification
The previously discussed isosurface and surface simplification issues
and tradeoffs apply to both of these approaches. In addition, the following
questions arise: How well does the resulting surface actually
"fit" the original contours? What is the best way to measure and
express the possible or actual error? Is the error data-dependent? Such
error can be important when viewing the tumor surface.
In general, RTP volume data can exist in one of three forms:
- A completely regular grid with regular positions and
connections (e.g., a set of CTs at constant table
position increment, each with identical XY resolution and
field of view [FOV]);
- A stack of completely regular and identical XY grids, but
at variable Z spacing (e.g., the previous CT set, but at
nonuniform table positions); and
- A stack of completely regular but different XY grids, at
variable or constant Z spacing (e.g., the previous CT
set, but with a different FOV or resolution for some
For certain volume data operations, regular grids can have much lower
memory requirements and can be substantially faster to work with then
irregular grids. Such operations include mapping the volume data to
arbitrary planes or surfaces and isosurface generation. This can give
incentive to regularize the volume data on input, for example, by remapping
all CT or dose planes to the same XY resolution and FOV, or by remapping
variably spaced CT or dose planes to a constant Z spacing. The following
analysis for CTs applies to any volume data.
In remapping XY, there are two choices. To reduce information loss, all
images might be remapped to the pixel size of the highest resolution image.
This can cause a data explosion. Consider a stack of 512x512 CTs. If just
one CT has an FOV of 1/2 the others, remapping would produce a stack of
1024x1024 CTs. However, if the smaller FOV CTs are not of interest, they
could simply be removed from the data set, with no remapping required. To
prevent such data growth, one can instead remap to the pixel size and FOV
of the lowest resolution image. This would cause loss of information in the
CTs with 1/2 the FOV. This would be a nonissue if those CTs are not of
interest but could be critical if the higher resolution CTs were
specifically obtained for more detail in the target area. Note, even when
remapping to the smallest pixel size, interpolation will occur and
information can still be lost.
In remapping Z, the issues are similar. To reduce information loss, one
might interpolate CTs at regular Z spacing equal to the minimal Z spacing
of the original set. This can also cause a data explosion. Consider one CT
at 1/3 the spacing of all the others. Remapping would triple the data
size. In any case, even if most of the original CTs are regularly spaced,
it is quite possible that most of the interpolated set will not align with
them, resulting in loss of information. On the other hand, if the
irregularly spaced CTs are not of interest, they could simply be removed,
with no need to regularize.
The impact of the chosen approach and its significance depend entirely
on (1) the structure of the original data, (2) the target of the
visualization, and (3) the goals of the visualization. Perhaps both the
original and regularized data should be carried along, each for different
purposes. The observer is usually unaware of these issues and thus has no
idea of the accuracy or meaning of what is being observed.
Voxel versus Point Values
Whether volume data is interpreted as voxel data or point data has an
impact on how well various visualizations will agree. For example, dose
isosurfaces are usually interpolated through a volume of dose points.
However, in DVH calculations, dose voxels might be employed, where the dose
pixel is considered constant through the slice thickness. Now consider an
ideal structure, precisely filled with 100% dose voxels, with 0% dose
voxels outside. The DVH will indicate that 100% of the structure receives
100% dose. However, a 100% dose isosurface will lie entirely inside the
displayed structure surface, since it will pass through the 100% dose
points, and not at the boundaries of the 100% dose voxels. A 50% dose
isosurface might be required to fully "cover" the structure. The
same problem can occur in mapping dose values onto the structure surface,
since each surface point will get an interpolated value between the inside
and outside dose point, not the value of the dose voxel it passes through.
Most of the surface might appear to receive 50% dose.
To bring these into agreement, one can employ voxel data in the
isosurface and dose mapping steps. The dose isosurface would be the
boundary of the dose voxels (like stacked sugar cubes). This would require
something other than marching cubes and would create a much more complex
surface (causing reduced interactivity). It would also be difficult to
correctly display voxel dose data mapped onto the structure surface, given
the triangulated surface facets.
The mapping of data onto a color scale should be tuned to the goal of
the visualization, the type of data and its spatial frequency, and human
visual perception if we expect the user to accurately interpret the
information . For example, the commonly used
"rainbow" color map both creates erroneous information (values
appear to be clustered in discrete bands) and hides real information
(hiding detail within those bands). Allowing the user to create custom
color maps is equally if not more dangerous. When observing apparent
structure in color mapped data (such as dose values), does this represent
the data, or is it just a color mapping artifact? One solution is described
in , which provides optimal color maps based
on data type and spatial frequency, the user's task, and properties of the
human perceptual system. We need to characterize the color maps most
appropriate for common RTP visualization tasks.
How missing data is handled is important. Interpolation can lead to
misinterpretation of data, but leaving it blank, black, or transparent can
distract the viewer from more relevant information. In , a color map is used for valid data, a grayscale
for missing data, with the boundaries matched via luminance value. This
does not distract the viewer at the edges, yet clearly shows valid and
A Proposed Model
A method of modeling and evaluating the RTP visualization issues is now
presented. Such a tool could be utilized to assess tradeoffs during RTP
system development, to compare different RTP systems, or to research
optimal visualization techniques. This is a strawman proposal, offered
primarily to help understand and structure the issues. In the end it may
serve only to show how not to do this. There are many related efforts at
modeling and evaluation in general visualization, computer graphics, image
processing, and computer vision, such as [12 ,
13 , 5 , 14 , 7]. This proposal
is a first step in applying some of these ideas to the RTP problem, but
much more investigation is required.
Modeling the Visualization Process
An RTP system can be viewed as two cascaded processes:
- the creation of a treatment plan, followed by
- the visualization of that plan.
The first process is outside the scope of this paper, but its output
data, such as the dose volume, reference points, beam parameters, and block
design, form the input to the visualization process.
The visualization process is modeled in an abstract, object-oriented
manner (e.g., ). Each large-grain
visualization operation becomes an object with well-defined inputs,
outputs, and data transformations (transforms). For example, an isosurface
object might take volume data and isovalue as inputs, produce a
triangulated surface as output, with the marching cubes algorithm as the
transform. Since the input and output data are objects as well,
"operation" objects will generally be called modules, such as an
IsoSurface module or SurfaceSimplify module. Note, the final visualization
module will usually be the HumanObserver.
Each visualization module must also provide some measure of error,
confidence, or quality of the output data. This is not necessarily a simple
value. For example, a SurfaceSimplify module would provide indication of
how the new surface relates to the original. As in  this would be
- a guarantee that volume is preserved,
- a guarantee of maximum vertex error, and
- an absolute error value for each vertex.
This quality measure will be termed the Quality Metric (QM). The effect
of QMs must be propagated through all modules in the visualization process.
Each module must calculate its output QM by considering the QM of its
inputs and how these are impacted by its own transforms. It will then be
possible to measure, evaluate, and compare the results of
For general scientific visualization efforts, this type of modeling is a
formidable task and not well-understood, especially considering how such
"quality" issues compound, cancel, or otherwise interact (e.g.,
[5 , 14]). However,
the RTP problem is very well-bounded with a rather small set of clearly
defined goals, hopefully making it more amenable to such a solution. The
process of developing this model will be iterative and evolutionary. As
modules are defined, the need to modify or create new data types or QMs may
arise. As data types or QMs change, previously defined modules may have to
be modified. The overriding constraint must be to keep things as simple as
possible, but no simpler (paraphrasing Einstein).
The necessary modeling tasks are now briefly introduced, focusing
primarily on the more difficult QM modeling.
Transforms (data transformations) are modeled at a high level of
abstraction. For example, a Slice module which extracts a 2D slice from 3D
data by holding one element of the position vector constant would be
trivial to model. Similarly, a spatial filtering module can be fully
characterized by its impulse response or a simplified approximation.
However, consider an IsoSurface module based on marching cubes. How can the
resulting surface be modeled in a simple way? What really needs to be
modeled? Following are some examples of the issues.
How large is the surface? This information is necessary to determine
what can be resolved on this surface, possibly based on the number of
display pixels that render it, or the angle it subtends in the human visual
FOV. For example, an isosurface accurate to 1% may be of little value if
the end observer can only detect a delta of 10% in the surface
What size are the triangles? Triangle size is necessary to understanding
the resolution of data mapped onto this surface. For example, mapping a
high resolution dose volume matrix onto triangles that are 10 times the
size of the dose voxels can result in substantial information loss, even
though smooth shading among vertices would hide this.
Other possibly important aspects include number of triangles, surface
normals, whether data is position (vertex)-dependent or connection
(triangle)-dependent, and data type. These issues must be thought out for
each relevant visualization operation.
Data Type Modeling
This is the simplest modeling issue and would proceed in parallel with
transform and QM modeling. Fundamental data types can be seen in any of the
common visualization packages, and from this, simplified versions for our
purposes can be created.
Quality Metric (QM) Modeling
Whereas the transform describes the changes to the data, the QM provides
ancillary information about the quality of that data (e.g., an error bound
on a data set or on each data element or other descriptive information). QM
information will exist purely as attributes on the transform results. One
should be able to remove all QM information and still have a workable model
(though far less useful), whereas removing the transform information would
break the model.
Since each transform has an impact on the QM of its outputs, the value
of the QM is clearly transform-dependent. However, the structure of the QM
must be purely data-dependent. This allows each module to process the QM
independently, regardless of where it comes from. Following are some issues
that the QM must handle, as well as some examples.
Non-information that is added. This could be due to smooth shading,
color mapping, interpolation across missing data, or interpolation at
higher resolution than original data.
Information that is lost. This could result from smooth shading and its
relation to viewing transformations and lighting, color mapping, data
filtering, interpolation, or masking by non-information.
Error bounds on valid information. These are often based on a
mathematical transform (e.g., calculation error due to the algorithm or
data type limitations). However, for the HumanObserver module, they will be
based on visual perception issues.
The intrinsic value of a visualization method. Different methods will
have different impact and value. For example, displaying a dose volume
matrix as raw numerical data is completely accurate but worthless as a
visualization tool. Displaying it as an isosurface is more informative but
perhaps less accurate. Visualization methods could be evaluated and ranked
in terms of suitability [14 , 5], based on data characteristics, goals of the
visualization, required accuracy, and visual perception issues.
Following are some examples. Consider the process of regularizing a dose
volume data set (discussed in the first part of this paper). The QM should
indicate how the regularized data differ from the original. This cannot be
expressed as a simple value. Consider that 44 out of 45 planes might align
exactly with the originals, with just one that is interpolated between two
originals. If it lies on the edge of the study, it might not matter. If it
lies in the tumor volume, it might matter a great deal. This information
can be captured in the QM by assigning an error value per plane; 44 planes
have zero error, one plane has non-zero error. Later analysis of the
particular visualization tasks and targets will then determine the
significance of this error.
As another example, consider a dose isosurface generated from marching
cubes. Each facet represents the physical location of that dose value and
corresponds to a dose voxel. If this surface is simplified, the resulting
facets still represent the location of that dose value, but now there are
possible positional errors. The QM can represent this with a position
error, but what does that say about the possible dose error? One really
wants to know, for a given facet representing a particular dose value, how
much that dose can be in error. This might require mapping the position
error back to the original dose matrix and using its gradient to determine
an error bound on the dose value represented by the isosurface. Further,
these new facets no longer correspond to original dose voxels. The QM could
include an attribute that specifies relationship of facets to input data.
Prior to simplification, the facet relationship would indicate input
voxels. After simplification, it would be null. If a subsequent
visualization employed flat shading to show original voxel size, this
attribute would signal that the facets no longer represented anything
Understanding how to propagate QMs, how they compound, cancel, or
otherwise interact can be tricky. Simply changing the order of operations
can have a big impact. For example, interpolating between vertex colors in
rendering hardware can give different results than interpolating data
values and then mapping to colors . These
issues have to be thought out.
Example Use of the Model
In the end, if the accuracy requirements of the final data
interpretation can be specified, the modeling tool will determine if a
proposed visualization will suffice. A simple scenario will demonstrate
this. Assume the goals are to
- visualize dose within a specified accuracy and
- to resolve dose changes of a particular delta.
One visualization of this might consist of
- spatially filtering the original dose data,
- generating an isosurface,
- smooth shading,
- displaying, and finally
The analysis would determine the error bound of the dose value
represented by the resulting isosurface by considering steps (1) through
(4). Even if this error is within the desired tolerance, the process is not
complete. The analysis must also consider the impact of display and
observation of that isosurface, such as whether the viewing angle and pixel
resolution are sufficient for the observer to detect the desired changes.
Another visualization might consist of
- generating a structure surface from contours,
- mapping dose data onto it,
- applying a color map,
- smooth shading,
- displaying, and
Here, an additional constraint might specify that the structure surface
align with the original contours within some tolerance. The surface error
bound would be determined from steps (1) and (2), with the dose error bound
determined from steps (3) through (5). Then, as above, consideration must
be given to the impact of display and observation, such as whether the
color map will allow the observer to detect the desired dose changes.
The issue of accuracy in RTP 3D visualization techniques was presented,
pointing to the need for methods of measurement and evaluation. A number of
common RTP visualization issues were analyzed in an initial attempt to
characterize the problem. A strawman proposal was then presented,
describing one approach to modeling and evaluating these issues. Only the
surface of these issues has been scratched. The goal was to initiate
discussion on this problem and possible solutions.
 Marchak et al. The psychology of
visualization. In IEEE Visualization 93,
pages 351-54, 1993.
 Twiddy et al. Restorer: A visualization
technique for handling missing data. In IEEE
Visualization 94, pages 212-16, 1994.
 Bergman et al. A rule-based tool for
assisting colormap selection. In IEEE Visualization
95, pages 118-25, 1995.
 Roux et al. Visualization in medicine:
Virtual reality or actual reality? In IEEE
Visualization 94, pages 396-99, 1994.
 Globus et al. Evaluation of visualization
software. Computer Graphics, 29(2):41-4,
 Gershon. Breaking the myth: One picture is
not (always) worth a thousand words. In IEEE
Visualization 96, pages 441-43, 1996.
 Rushmeier et al. Metrics and benchmarks
for visualization. In IEEE Visualization 95,
pages 422-26, 1995.
 Lorensen et al. Marching cubes: A high
resolution 3d surface construction algorithm. Computer
Graphics, 21(3):163-69, July 1987.
 Shekhar et al. Octree-based decimation of
marching cubes surfaces. In IEEE Visualization 96,
pages 335-42, 1996.
 Gueziec. Surface simplification inside a
tolerance volume. Technical Report RC 20440, IBM,
 Boissonnat et al. Three dimensional
reconstruction of complex shapes based on the delaunay
triangulation. In Biomedical Image Processing and
Biomedical Visualization, pages 964-75, 1993.
 Butler et al. Visualization reference
models. In IEEE Visualization 93, pages
 Hildebrand et al. Towards a visual
computing and communication reference model. Computers
& Graphics, 19(1):144-49, 95.
 Arndt et al. Design of a visualization
support tool for the representation of multi-dimensional data
sets. In Visualization in Scientific Computing,
pages 190-204, 1995.
Back to top