Scholars@Duke
Josef Spjut

Josef Spjut

Adjunct Associate Professor in the Engineering Graduate and Professional Programs
Engineering Graduate and Professional Programs

Selected Publications

Esports Rendering: Industry and Player Perspectives

Conference Proceedings SIGGRAPH 2025 Frontiers · August 19, 2025 Esports is a unique challenge for rendering research, with players regularly turning off even basic rendering techniques to reduce latency. In this workshop, three esports developers and three competing esports athletes will form an expert panel on esports ... Full text Cite

Assessing Learned Models for Phase-only Hologram Compression

Conference Proceedings SIGGRAPH 2025 Posters · August 10, 2025 We evaluate the performance of four common learned models utilizing INR and VAE structures for compressing phase-only holograms in holographic displays. The evaluated models include a vanilla MLP, SIREN [Sitzmann et al. 2020], and FilmSIREN [Chan et al. 20 ... Full text Cite

Toward Understanding Display Size for FPS Esports Aiming

Conference Proceedings of the 20th International Conference on the Foundations of Digital Games Fdg 2025 · May 9, 2025 Gamers use a variety of different display sizes, though for PC gaming, monitors in the 24 to 27 inch size range have become most popular. Particularly popular among many PC gamers, first person shooter (FPS) games represent a genre where hand-eye coordinat ... Full text Cite

Modeling visually-guided aim-and-shoot behavior in first-person shooters

Journal Article International Journal of Human Computer Studies · May 1, 2025 In first-person shooters, players aim by aligning the crosshair onto a target and shoot at the optimal moment. Since winning a match is largely determined by such aim-and-shoot skills, players demand quantitative evaluation of the skill and analysis of hid ... Full text Cite

Pushing the Limits? Frame Rate Benefits to Players for up to 500 Hz in First Person Shooter Games

Conference Nossdav 2025 Proceedings of the 2025 35th Edition of the Workshop on Network and Operating System Support for Digital Audio and Video · April 3, 2025 Computer games-and computer game players-often drive technology improvements, with graphics cards and monitors pushing the limits of display technologies. High frame rates, in particular, promise to provide lower latencies and smoother game visuals to game ... Full text Cite

eXtended Reality and Artificial Intelligence in Medicine and Rehabilitation

Journal Article Information Systems Frontiers · February 1, 2025 This special issue focuses on the application of eXtended Reality (XR) technologies—comprising Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR)—and Artificial Intelligence (AI) in the fields of medicine and rehabilitation. AR provides s ... Full text Cite

A Generative Ai Game Jam Case Study From October 2024

Conference IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · January 1, 2025 Generative Artificial Intelligence (GenAI) promises to democratize many creative endeavors, from art, to music, to writing. However, video games are an underexplored field for GenAI given the highly multi-modal and interactive nature. In this work, we pres ... Full text Cite

GRS: Generating Robotic Simulation Tasks from Real-World Images

Conference IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · January 1, 2025 We introduce GRS (Generating Robotic Simulation tasks), a system addressing real-to-sim for robotic simulations. GRS creates digital twin simulations from single RGB-D obser-vations with solvable tasks for virtual agent training. Using vision-language mode ... Full text Cite

Timing Matters: The Impact of Event-Specific Frametime Spikes in First-Person Shooter Games

Conference 2025 17th International Conference on Quality of Multimedia Experience Qomex 2025 · January 1, 2025 Frametime spikes can disrupt gameplay in first-person shooter (FPS) games, affecting both performance and player experience. This paper examines how spikes during specific game events impact players. We developed a custom FPS game that maintains a steady 5 ... Full text Cite

Lead Rush: A First-Person Shooter for User Studies and Understanding Effects of Frame Time Spikes

Conference 2025 17th International Conference on Quality of Multimedia Experience Qomex 2025 · January 1, 2025 User studies are a cornerstone of human-computer interaction research, including measures of user performance and quality of experience (QoE) - particularly important for games where frame rates and frame timings can impact performance. Unfortunately, comm ... Full text Cite

SpecTrack: Learned Multi-Rotation Tracking via Speckle Imaging

Conference Proceedings SIGGRAPH Asia 2024 Posters SA 2024 · December 3, 2024 Precision pose detection is increasingly demanded in fields such as personal fabrication, Virtual Reality (VR), and robotics due to its critical role in ensuring accurate positioning information. However, conventional vision-based systems used in these sys ... Full text Cite

On Smoothly Varying Frame Timing in First-Person Gaming

Conference Proceedings SIGGRAPH 2024 Talks · July 18, 2024 Full text Cite

The Effects of Network Latency on the Peeker's Advantage in First-person Shooter Games

Conference ACM International Conference Proceeding Series · May 21, 2024 In first-person shooter (FPS) games, the peeker's advantage is the edge the moving peeker gets when battling a stationary defender at a corner due to network latency. However, confirmation of (the size of) this advantage based on network latency and the di ... Full text Cite

Department: Applications Is Less More Rendering for Esports

Journal Article IEEE Computer Graphics and Applications · March 1, 2024 Computer graphics research has long prioritized image quality over frame rate. Yet demand for an alternative is growing, with many esports players turning off visual effects to improve frame rates. Is it time for graphics researchers to reconsider their go ... Full text Cite

Variable Frame Timing Affects Perception of Smoothness in First-Person Gaming

Conference IEEE Conference on Computatonal Intelligence and Games Cig · January 1, 2024 With the advent of variable refresh rate (VRR) monitor technologies, gamers experience variable frame timing (VFT) during their gameplay. Combining VRR with low-latency GPU rendering and increased display refresh rates enables smoother variation of frame p ... Full text Cite

Mouse Sensitivity in First-Person Targeting Tasks

Journal Article IEEE Transactions on Games · December 1, 2023 Mouse sensitivity in first-person targeting tasks is a highly debated issue. Recommendations within a single game can vary by a factor of 10× or more and are an active topic of experimentation in both competitive and recreational esports communities. Inspi ... Full text Cite

Rethinking Display Requirements for Esports and High Interactivity Applications

Conference Digest of Technical Papers SID International Symposium · January 1, 2023 Media technology is continuing its transition from passive streaming to participatory interactive experiences, including well-known applications such as web browsing, video conferencing and gaming, as well as emerging and more demanding uses like AR/MR/VR ... Full text Cite

Constant Field of View Display Size Effects on First-Person Aiming Time

Conference Frontiers in Optics Proceedings Frontiers in Optics Laser Science 2023 Fio Ls 2023 · January 1, 2023 Under constant display field of view, FPS game aiming performance improves with display size, resulting in 3% faster aiming time comparing 13 and 26 inches diagonal. ... Full text Cite

Display Size and Targeting Performance: Small Hurts, Large May Help

Conference Proceedings SIGGRAPH Asia 2022 Conference Papers · November 29, 2022 Which display size helps gamers win? Recommendations from the research and PC gaming communities are contradictory. We find that as display size grows, targeting performance improves. When size increases from 13′′ to 26′′, targeting time drops by over 3%. ... Full text Cite

Esports and expertise

Journal Article Interactions · November 3, 2022 Historically, much research and development in human-computer interaction has focused on atomic and generalizable tasks, where task completion time indicates productivity. The emergence of competitive games and esports, however, reminds us of an alternativ ... Full text Cite

FirstPersonScience: An Open Source Tool for Studying FPS Esports Aiming

Conference Proceedings SIGGRAPH 2022 Talks · July 24, 2022 First-person shooters (FPS) games are dominant in the competitive gaming and esports community. However, relatively few tools are available for experimenters interested in studying mechanics of these games in a controlled, repeatable environment. While oth ... Full text Cite

Esports meets human-computer interaction

Journal Article Interactions · May 1, 2022 The number of researchers within the HCI community involved in esports-related work has increased recently. Multiple events have been organized to bring together researchers and industry professionals from different disciplines and promote interdisciplinar ... Full text Cite

Mouse Sensitivity in First-person Targeting Tasks

Conference IEEE Conference on Computatonal Intelligence and Games Cig · January 1, 2022 Despite billions of hours of play and copious discussion online, mouse sensitivity recommendations for first-person targeting tasks vary by a factor of 10x or more and remain an active topic of debate in both competitive and recreational gaming communities ... Full text Cite

Esports as a Driving Problem in Computer Graphics

Conference Special Interest Group on Computer Graphics and Interactive Techniques Conference SIGGRAPH 2021 Panels · September 22, 2021 Esports is a growing worldwide phenomenon now rivaling traditional sports, with a deep dependence on real-time graphics technology. Despite this, the SIGGRAPH research community has largely ignored it. This panel brings together esports experts in engineer ... Full text Cite

Gaming at Warp Speed: Improving Aiming with Late Warp

Conference Special Interest Group on Computer Graphics and Interactive Techniques Conference Emerging Technologies SIGGRAPH 2021 · August 6, 2021 Latency can make all the difference in competitive online games. Late warp is a class of techniques used in VR that can reduce latency in FPS games as well. Prior work has demonstrated these techniques can recover most of the player performance lost to com ... Full text Cite

Esports and High Performance HCI

Conference Conference on Human Factors in Computing Systems Proceedings · May 8, 2021 Competitive esports is a growing worldwide phenomenon now rivaling traditional sports, with over 450 million views and 1 billion US dollars in revenue each year. For comparison, Major League Baseball has 500 million views and $10 billion in revenue, FIFA S ... Full text Cite

Noise-Aware Video Saliency Prediction

Conference 32nd British Machine Vision Conference Bmvc 2021 · January 1, 2021 We tackle the problem of predicting saliency maps for videos of dynamic scenes. We note that the accuracy of the maps reconstructed from the gaze data of a fixed number of observers varies with the frame, as it depends on the content of the scene. This iss ... Cite

Post-Render Warp with Late Input Sampling Improves Aiming Under High Latency Conditions

Journal Article Proceedings of the ACM on Computer Graphics and Interactive Techniques · August 26, 2020 End-to-end latency in remote-rendering systems can reduce user task performance. This notably includes aiming tasks on game streaming services, which are presently below the standards of competitive first-person desktop gaming. We evaluate the latency-indu ... Full text Cite

Toward Standardized Classification of Foveated Displays.

Journal Article IEEE transactions on visualization and computer graphics · May 2020 Emergent in the field of head mounted display design is a desire to leverage the limitations of the human visual system to reduce the computation, communication, and display workload in power and form-factor constrained systems. Fundamental to this reduced ... Full text Cite

Latency of 30 ms benefits first person targeting tasks more than refresh rate above 60 Hz

Conference SIGGRAPH Asia 2019 Technical Briefs SA 2019 · November 17, 2019 In competitive sports, human performance makes the difference between who wins and loses. In some competitive video games (esports), response time is an essential factor of human performance. When the athlete’s equipment (computer, input and output device) ... Full text Cite

A Deformable Interface for Human Touch Recognition Using Stretchable Carbon Nanotube Dielectric Elastomer Sensors and Deep Neural Networks.

Journal Article Soft robotics · October 2019 This article presents a machine learning approach to map outputs from an embedded array of sensors distributed throughout a deformable body to continuous and discrete virtual states, and its application to interpret human touch in soft interfaces. We integ ... Full text Cite

Foveated displays: Toward classification of the emerging field

Conference ACM SIGGRAPH 2019 Talks SIGGRAPH 2019 · July 28, 2019 There is not yet consensus in the field on what constitutes a “foveated display”. We propose a compromise between the perspectives of rendering, imaging, physiology and vision science that defines a foveated display as a display designed to function in the ... Full text Cite

Matching prescription & visual acuity: Towards AR for humans

Conference ACM SIGGRAPH 2019 Emerging Technologies SIGGRAPH 2019 · July 28, 2019 Full text Cite

Foveated AR: Dynamically-foveated augmented reality display

Journal Article ACM Transactions on Graphics · July 1, 2019 We present a near-eye augmented reality display with resolution and focal depth dynamically driven by gaze tracking. The display combines a traveling microdisplay relayed off a concave half-mirror magnifier for the high-resolution foveal region, with a wid ... Full text Cite

SIDOD: A synthetic image dataset for 3D object pose recognition with distractors

Conference IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · June 1, 2019 We present a new, publicly-available image dataset generated by the NVIDIA Deep Learning Data Synthesizer intended for use in object detection, pose estimation, and tracking applications. This dataset contains 144k stereo image pairs that synthetically com ... Full text Cite

Fluidic Elastomer Actuators for Haptic Interactions in Virtual Reality

Journal Article IEEE Robotics and Automation Letters · April 1, 2019 Virtual reality experiences via immersive optics and sound are becoming ubiquitous; there are several consumer systems (e.g., Oculus Rift and HTC Vive) now available with these capabilities. Other sensory experiences, such as that of touch remain elusive i ... Full text Cite

UIBK: User interactions for building knowledge

Conference International Conference on Intelligent User Interfaces Proceedings IUI · March 16, 2019 This half-day workshop seeks to bring together practitioners and academics interested in the challenges of structuring interactions for subject matter experts (SMEs) who are providing knowledge and/or feedback to an AI system, but are not well-versed in th ... Full text Cite

Improving temporal antialiasing with adaptive ray tracing

Chapter · February 25, 2019 In this chapter, we discuss a pragmatic approach to real-time supersampling that extends commonly used temporal antialiasing techniques with adaptive ray tracing. The algorithm conforms to the constraints of a commercial game engine, removes blurring and g ... Full text Cite

Temporally dense ray tracing

Conference High Performance Graphics ACM SIGGRAPH Eurographics Symposium Proceedings Hpg · January 1, 2019 We present a technique for real-time ray tracing with the goal of reaching 240 frames per second or more. The core idea is to trade spatial resolution for faster temporal updates in such a way that the display and human visual system aid in integrating hig ... Full text Cite

Steerable application-adaptive near eye displays

Conference ACM SIGGRAPH 2018 Emerging Technologies SIGGRAPH 2018 · August 12, 2018 The design challenges of see-through near-eye displays can be mitigated by specializing an augmented reality device for a particular application. We present a novel optical design for augmented reality near-eye displays exploiting 3D stereolithography prin ... Full text Cite

Adaptive temporal antialiasing

Conference High Performance Graphics ACM SIGGRAPH Eurographics Symposium Proceedings Hpg · August 10, 2018 We introduce a pragmatic algorithm for real-time adaptive super-sampling in games. It extends temporal antialiasing of rasterized images with adaptive ray tracing, and conforms to the constraints of a commercial game engine and today's GPU ray tracing APIs ... Full text Cite

A variable shape and variable stiffness controller for haptic virtual interactions

Conference 2018 IEEE International Conference on Soft Robotics Robosoft 2018 · July 5, 2018 This paper presents an entirely compliant controller handle for use in virtual and augmented reality environments. The controller handle transitions between two static states: A semi-rigid, large diameter state when pneumatically pressurized and a soft, co ... Full text Cite

SimTRaX: Simulation infrastructure for exploring thousands of cores

Conference Proceedings of the ACM Great Lakes Symposium on VLSI Glsvlsi · May 30, 2018 SimTRaX is a simulation infrastructure for simultaneous exploration of highly parallel accelerator architectures and how applications map to them. The infrastructure targets both cycle-accurate and functional simulation of architectures with thousands of s ... Full text Cite

Varifocal virtuality: A novel optical layout for near-eye display

Conference ACM SIGGRAPH 2017 Emerging Technologies SIGGRAPH 2017 · July 30, 2017 Augmented reality (AR) has recently gained momentum in the form of a variety of available optical see-through near-eye displays (NEDs) such as the Meta 2and the Microsoft Hololens. These devices are a big step forward towards Sutherland's vision of an ulti ... Full text Cite

Stretchable transducers for kinesthetic interactions in virtual

Conference ACM SIGGRAPH 2017 Emerging Technologies SIGGRAPH 2017 · July 30, 2017 Commercial virtual reality devices such as the Oculus Rift and HTC Vive enable experience designers to emulate a number of human sensory inputs with computer simulations. Commercial experiences have demonstrated plausibly realistic audiovisual sensory inpu ... Full text Cite

Hybrid Modulation for Near Zero Display Latency

Conference Digest of Technical Papers SID International Symposium · January 1, 2016 Binary displays for virtual reality can achieve low latency by integrating view tracking with modulation. We present a novel modulation scheme that combines tracking, pulse density modulation, and pulse width modulation to minimize grayscale artifacts. The ... Full text Cite

Build your own game controller

Conference SIGGRAPH 2015 Studio SIGGRAPH 2015 · July 31, 2015 Full text Cite

Memory considerations for low energy ray tracing

Journal Article Computer Graphics Forum · February 1, 2015 We propose two hardware mechanisms to decrease energy consumption on massively parallel graphics processors for ray tracing. First, we use a streaming data model and configure part of the L2 cache into a ray stream memory to enable efficient data processin ... Full text Cite

An energy and bandwidth efficient ray tracing architecture

Conference Proceedings High Performance Graphics 2013 Hpg 2013 · January 1, 2013 We propose two hardware mechanisms to decrease energy consumption on massively parallel graphics processors for ray tracing while keeping performance high. First, we use a streaming data model and configure part of the L2 cache into a ray stream memory to ... Full text Cite

Fast, effective BVH updates for animated scenes

Conference Proceedings I3d 2012 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games · April 23, 2012 Bounding volume hierarchies (BVHs) are a popular acceleration structure choice for animated scenes rendered with ray tracing. This is due to the relative simplicity of refitting bounding volumes around moving geometry. However, the quality of such a refitt ... Full text Cite

Efficient MIMD architectures for high-performance ray tracing

Conference Proceedings IEEE International Conference on Computer Design VLSI in Computers and Processors · December 1, 2010 Ray tracing efficiently models complex illumination effects to improve visual realism in computer graphics. Typical modern GPUs use wide SIMD processing, and have achieved impressive performance for a variety of graphics processing including ray tracing. H ... Full text Cite

SWEL: Hardware cache coherence protocols to map shared data onto shared caches

Conference Parallel Architectures and Compilation Techniques Conference Proceedings Pact · January 1, 2010 Snooping and directory-based coherence protocols have become the de facto standard in chip multi-processors, but neither design is without drawbacks. Snooping protocols are not scalable, while directory protocols incur directory storage overhead, frequent ... Full text Cite

TRaX: A multicore hardware architecture for real-time ray tracing

Journal Article IEEE Transactions on Computer Aided Design of Integrated Circuits and Systems · December 1, 2009 Threaded Ray eXecution (TRaX) is a highly parallel multithreaded multicore processor architecture designed for real-time ray tracing. The TRaX architecture consists of a set of thread processors that include commonly used functional units (FUs) for each th ... Full text Cite

Hardware-accelerated gradient noise for graphics

Conference Proceedings of the ACM Great Lakes Symposium on VLSI Glsvlsi · November 6, 2009 A synthetic noise function is a key component of most computer graphics rendering systems. This pseudo-random noise function is used to create a wide variety of natural looking textures that are applied to objects in the scene. To be useful, the generated ... Full text Cite

Comparing incoherent ray performance of TRaX vs. Manta

Conference RT 08 IEEE Eg Symposium on Interactive Ray Tracing 2008 Proceedings · November 28, 2008 TRaX (Threaded Ray eXecution) is a highly parallel multi-threaded, multi-core processor architecture designed for real-time ray tracing. One motivation behind TRaX is to accelerate single-ray performance instead of relying on ray-packets in SIMD mode to bo ... Full text Cite

TRaX: A multi-threaded architecture for real-time ray tracing

Conference 2008 Symposium on Application Specific Processors Sasp 2008 · September 29, 2008 Ray tracing is a technique used for generating highly realistic computer graphics images. In this paper, we explore the design of a simple but extremely parallel, multi-threaded, multi-core processor architecture that performs real-time ray tracing. Our ar ... Full text Cite