Hyung-Kwon Ko

BLOG (Korean)

Currently I'm with KIXLAB at KAIST under the supervision of Prof. Juho Kim. I'm working on two HCI projects: 1) "Enabling Prototyping of AI-infused UIs with Task-level Specifications" , and 2) "Understanding Artists' Perception, Expectation, and Concern for Large-scale Text-to-Image Generative Models", aiming to submit them to CHI 2023 and IUI 2023, respectively.

I am broadly interested in Human-Computer Interaction, Human-Centered AI, and Information Visualization. I am eager to find important problems in diverse domains, and help people using fancy techniques (it does not have to be AI). My life-long goal is to found a company with a product that can innovate people's working paradigm, in turn, changing their lives more intelligent and convenient. I prefer not to draw a strict boundary between academia and industry.

I received my Master's degree from the Department of Computer Science and Engineering of Seoul National University studying Human-computer Interaction and Information Visualization. During my MS, I worked with SNU HCIL under the supervision of Prof. Jinwook Seo mostly on dimensionality reduction methods. I received my Bachelor's degree from Hanyang University. During my BS, I majored in mathematics, minored in industrial engineering, and earned 16 credits in computer science and engineering. I was a full-time research scientist at Naver Webtoon Corp.


Jun 2022 Two papers, UMATO and We-toon, are accepted to IEEE VIS 2022 and ACM UIST 2022, respectively!
Jun 2022 Moved to KAIST at Daejeon, Korea to collaborate with KIXLAB led by Prof. Juho Kim
Jan 2022 Published an arxiv paper about interactive brushing technique
Oct 2021 Got a full-time posiiton as a research scientist at Naver Webtoon Corp.
Apr 2021 Graduated from Seoul National University receiving Master's degree in Computer Science and Engineering under the supervision of Prof. Jinwook Seo (specialty: HCI, InfoVis)


We-toon: A Communication Support System between Writers and Artists in Collaborative Webtoon Sketch Revision
Hyung-Kwon Ko*, Subin An*, Gwanmo Park, Seung Kwon Kim, Daesik Kim, Bohyoung Kim, Jaemin Jo, Jinwook Seo
UIST 2022 (to appear)

We present a communication support system, namely We-toon, that can bridge the webtoon writers and artists during sketch revision (i.e., character design and draft revision). In the highly iterative design process between the webtoon writers and artists, writers often have difficulties in precisely articulating their feedback on sketches owing to their lack of drawing proficiency. This drawback makes the writers rely on textual descriptions and reference images found using search engines, leading to indirect and inefficient communications. Inspired by a formative study, we designed We-toon to help writers revise webtoon sketches and effectively communicate with artists. Through a GAN-based image synthesis and manipulation, We-toon can interactively generate diverse reference images and synthesize them locally on any user-provided image. Our user study with 24 professional webtoon authors demonstrated that We-toon outperforms the traditional methods in terms of communication effectiveness and the writers’ satisfaction level related to the revised image.

Uniform Manifold Approximation with Two-phase Optimization
Hyeon Jeon*, Hyung-Kwon Ko*, Soohyun Lee, Jaemin Jo, Jinwook Seo
VIS 2022 short (to appear)
paper / code / video

We introduce Uniform Manifold Approximation with Two-phase Optimization (UMATO), a dimensionality reduction (DR) technique that improves UMAP to capture the global structure of high-dimensional data more accurately. In UMATO, optimization is divided into two phases so that the resulting embeddings can depict the global structure reliably while preserving the local structure with sufficient accuracy. As the first phase, hub points are identified and projected to construct a skeletal layout for the global structure. In the second phase, the remaining points are added to the embedding preserving the regional characteristics of local areas. Through quantitative experiments, we found that UMATO (1) outperformed widely used DR techniques in preserving the global structure while (2) producing competitive accuracy in representing the local structure. We also verified that UMATO is preferable in terms of robustness over diverse initialization methods, number of epochs, and subsampling techniques.

Distortion-Aware Brushing for Interactive Cluster Analysis in Multidimensional Projections
Hyeon Jeon, Michael Aupetit, Soohyun Lee, Hyung-Kwon Ko, Youngtaek Kim, Jinwook Seo
Arxiv 2022 (Under review)
paper / code

Brushing is an everyday interaction in 2D scatterplots, which allows users to select and filter data points within a continuous, enclosed region and conduct further analysis on the points. However, such conventional brushing cannot be directly applied to Multidimensional Projections (MDP), as they hardly escape from False and Missing Neighbors distortions that make the relative positions of the points unreliable. To alleviate this problem, we introduce Distortion-aware brushing, a novel brushing technique for MDP. While users perform brushing, Distortion-aware brushing resolves distortions around currently brushed points by dynamically relocating points in the projection; the points whose data are close to the brushed data in the multidimensional (MD) space go near the corresponding brushed points in the projection, and the opposites move away. Hence, users can overcome distortions and readily extract out clustered data in the MD space using the technique. We demonstrate the effectiveness and applicability of Distortion-aware brushing through usage scenarios with two datasets. Finally, by conducting user studies with 30 participants, we verified that Distortion-aware brushing significantly outperforms previous brushing techniques in precisely separating clusters in the MD space, and works robustly regardless of the types or the amount of distortions in MDP.

Measuring and explaining the inter-cluster reliability of multidimensional projections
Hyeon Jeon, Hyung-Kwon Ko, Jaemin Jo, Youngtaek Kim, Jinwook Seo
TVCG 2021 (Proc. VIS 2021)
paper / code / video / bibtex / doi / post

We propose Steadiness and Cohesiveness, two novel metrics to measure the inter-cluster reliability of multidimensional projection (MDP), specifically how well the inter-cluster structures are preserved between the original high-dimensional space and the low-dimensional projection space. Measuring inter-cluster reliability is crucial as it directly affects how well inter-cluster tasks (e.g., identifying cluster relationships in the original space from a projected view) can be conducted; however, despite the importance of inter-cluster tasks, we found that previous metrics, such as Trustworthiness and Continuity, fail to measure inter-cluster reliability. Our metrics consider two aspects of the inter-cluster reliability: Steadiness measures the extent to which clusters in the projected space form clusters in the original space, and Cohesiveness measures the opposite. They extract random clusters with arbitrary shapes and positions in one space and evaluate how much the clusters are stretched or dispersed in the other space. Furthermore, our metrics can quantify pointwise distortions, allowing for the visualization of inter-cluster reliability in a projection, which we call a reliability map. Through quantitative experiments, we verify that our metrics precisely capture the distortions that harm inter-cluster reliability while previous metrics have difficulty capturing the distortions. A case study also demonstrates that our metrics and the reliability map 1) support users in selecting the proper projection techniques or hyperparameters and 2) prevent misinterpretation while performing inter-cluster tasks, thus allow an adequate identification of inter-cluster structure.

Mixed-Initiative Approach to Extract Data from Pictures of Medical Invoice
Seokweon Jung, Kiroong Choe, Seokhyeon Park, Hyung-Kwon Ko, Youngtaek Kim, Jinwook Seo
PacificVis 2021 short
paper / video / bibtex / doi

Extracting data from pictures of medical records is a common task in the insurance industry as the patients often send their medical invoices taken by smartphone cameras. However, the overall process is still challenging to be fully automated because of low image quality and variation of templates that exist in the status quo. In this paper, we propose a mixed-initiative pipeline for extracting data from pictures of medical invoices, where deep-learning-based automatic prediction models and task-specific heuristics work together under the mediation of a user. In the user study with 12 participants, we confirmed our mixed-initiative approach can supplement the drawbacks of a fully automated approach within an acceptable completion time. We further discuss the findings, limitations, and future works for designing a mixed-initiative system to extract data from pictures of a complicated table.

Progressive Uniform Manifold Approximation and Projection
Hyung-Kwon Ko, Jaemin Jo, Jinwook Seo
EuroVis 2020 short
paper / code / video / bibtex / doi

We present a progressive algorithm for the Uniform Manifold Approximation and Projection (UMAP), called the Progressive UMAP. Based on the theory of Riemannian geometry and algebraic topology, UMAP is an emerging dimensionality reduction technique that offers better versatility and stability than t-SNE. Although UMAP is also more efficient than t-SNE, it still suffers from an initial delay of a few minutes to produce the first projection, which limits its use in interactive data exploration. To tackle this problem, we improve the sequential computations in UMAP by making them progressive, which allows people to incrementally append a batch of data points into the projection at the desired pace. In our experiment with the Fashion MNIST dataset, we found that Progressive UMAP could generate the first approximate projection within a few seconds while also sufficiently capturing the important structures of the high-dimensional dataset.


Abstract Sketch to Character
Spring 2022

Convert human-drawn abstract sketch image to an anime character with varying styles

Playing Atari with DQN
Winter 2019
code / video

Implemented DQN algorithm (Mnih et al. 2013) that can play one of the Atari games, 'Pong', and can beat the opponent

Musical Structure Visualization with MIDI Data
Fall 2019
demo / code / video

Information Visualization project at SNU

Korean Web Novel Generation
Spring 2018 - Summer 2018
paper / code / wiki / link / media

Made deep learning-based Korean sentence recommender. Received 20,000,000 KRW of prize money.


I love playing piano. The songs I used to play are this and this. Also, I like swimming a lot, so I go swimming almost everyday. I am a big fan of Tottenham Hotspur since 2009. I own more than 400 comic books and like reading Murakami Haruki's short novels. I love this quote the most:

Dance like nobody's watching; Love like you've never been hurt; Work like you don't need money; Live like it's heaven on earth.


Referred to existing websites such as this and this.