Kwon Ko

I am a Computer Science Ph.D. student at Stanford University, and my studies will be partially supported by the Fulbright Fellowship.

Please feel free to connect with me here if you'd like to chat.

I studied physics (BS, dropout), math (BS), industrial engineering (BS, minor), and computer science (MS).

Now I am studying neuroscience because I want to know how brain works.

Previously I worked at ROK Army (21 months), broadcasting company (7 months), consulting firm (3 months), manufacturing company (2 months), big tech company (15 months), and IT startup (5 months).

Sep 2024 Joined Stanford University for my Ph.D. studies in Computer Science
May 2024 Started working as a research engineer at SkillWave, Inc.
Apr 2024 Invited for a talk at Sungkyunkwan University and Seoul National University
Jan 2024 Chart-LLM is accepted to CHI 2024
Sep 2023 Nominated as Fulbright scholarship recipient
Aug 2023 Two papers are accepted to VIS 2023 and NLVIZ Workshop, respectively!
Mar 2023 ExGPTer is accepted to CHI 2023 Gen-AI workshop!
Jan 2023 Our paper about DALL-E is accepted to IUI 2023!
Nov 2022 My interview with IEEE Computer Society was published!
Oct 2022 Attend UIST 2022 in person to present my work, perform SV-ing, and give a lightning talk! Received Gary Marsden Travel Awards to support my trip.
Oct 2022 Submit two papers about DALL-E and generative model's disentanglement to IUI 2023.
Jun 2022 Two papers, UMATO and We-toon, are accepted to IEEE VIS 2022 and ACM UIST 2022, respectively!
Jun 2022 Move to KAIST at Daejeon, Korea to collaborate with KIXLAB led by Prof. Juho Kim.
Jan 2022 Publish an Arxiv paper about interactive brushing technique.
Oct 2021 Get a full-time posiiton as a research scientist at Naver Webtoon Corp.
Apr 2021 Graduate from Seoul National University receiving Master's degree in Computer Science and Engineering under the supervision of Prof. Jinwook Seo (specialty: HCI, InfoVis)

PUBLICATIONS

ChatGPT in Data Visualization Education: A Student Perspective
Nam Wook Kim, Kwon Ko, Grace Myers, Benjamin Bach
VL/HCC 2024
paper

Unlike traditional educational chatbots that rely on pre-programmed responses, large-language model-driven chatbots, such as ChatGPT, demonstrate remarkable versatility to serve as a dynamic resource for addressing student needs from understanding advanced concepts to solving complex problems. This work explores the impact of such technology on student learning in an interdisciplinary, project-oriented data visualization course. Throughout the semester, students engaged with ChatGPT across four distinct projects, designing and implementing data visualizations using a variety of tools such as Tableau, D3, and Vega-lite. We collected conversation logs and reflection surveys after each assignment and conducted interviews with selected students to gain deeper insights into their experiences with ChatGPT. Our analysis examined the advantages and barriers of using ChatGPT, students' querying behavior, the types of assistance sought, and its impact on assignment outcomes and engagement. We discuss design considerations for an educational solution tailored for data visualization education, extending beyond ChatGPT's basic interface.

Natural Language Dataset Generation Framework for Visualizations Powered by Large Language Models
Kwon Ko, Hyeon Jeon, Gwanmo Park, Dae Hyun Kim, Nam Wook Kim, Juho Kim, Jinwook Seo
CHI 2024
paper / code / project-page

We introduce VL2NL, a Large Language Model (LLM) framework that generates rich and diverse NL datasets using only Vega-Lite specifications as input, thereby streamlining the development of Natural Language Interfaces (NLIs) for data visualization. To synthesize relevant chart semantics accurately and enhance syntactic diversity in each NL dataset, we leverage 1) a guided discovery incorporated into prompting so that LLMs can steer themselves to create faithful NL datasets in a self-directed manner; 2) a score-based paraphrasing to augment NL syntax along with four language axes. We also present a new collection of 1,981 real-world Vega-Lite specifications that have increased diversity and complexity than existing chart collections. When tested on our chart collection, VL2NL extracted chart semantics and generated L1/L2 captions with 89.4% and 76.0% accuracy, respectively. It also demonstrated generating and paraphrasing utterances and questions with greater diversity compared to the benchmarks. Last, we discuss how our NL datasets and framework can be utilized in real-world scenarios. The codes and chart collection are available at this https URL.

Zadu: A python library for evaluating the reliability of dimensionality reduction embeddings
Hyeon Jeon, Aeri Cho, Jinhwa Jang, Soohyun Lee, Jake Hyun, Kwon Ko, Jaemin Jo, Jinwook Seo
VIS 2023 short
paper / code

Dimensionality reduction (DR) techniques inherently distort the original structure of input high-dimensional data, producing imperfect low-dimensional embeddings. Diverse distortion measures have thus been proposed to evaluate the reliability of DR embeddings. However, implementing and executing distortion measures in practice has so far been time-consuming and tedious. To address this issue, we present ZADU, a Python library that provides distortion measures. ZADU is not only easy to install and execute but also enables comprehensive evaluation of DR embeddings through three key features. First, the library covers a wide range of distortion measures. Second, it automatically optimizes the execution of distortion measures, substantially reducing the running time required to execute multiple measures. Last, the library informs how individual points contribute to the overall distortions, facilitating the detailed analysis of DR embeddings. By simulating a real-world scenario of optimizing DR embeddings, we verify that our optimization scheme substantially reduces the time required to execute distortion measures. Finally, as an application of ZADU, we present another library called ZADUVis that allows users to easily create distortion visualizations that depict the extent to which each region of an embedding suffers from distortions.

Moderating Customer Inquiries and Responses to Alleviate Stress and Reduce Emotional Dissonance of Customer Service Representatives
Kwon Ko, Kihoon Son, Hyoungwook Jin, Yoonseo Choi, Xiang Anthony Chen
CHI 2023 Generative AI and HCI Workshop
paper

Customer service representatives (CSRs) face significant levels of stress as a result of handling disrespectful customer inquiries and the emotional dissonance that arises from concealing their true emotions to provide the best customer experience. To solve this issue, we propose ExGPTer that uses ChatGPT to moderate the tone and manner of a customer inquiry to be more gentle and appropriate, while ensuring that the content remains unchanged. ExGPTer also augments CSRs' responses to answer customer inquiries, so they can conform to established company protocol while effectively conveying the essential information that customers seek.

Large-scale Text-to-Image Generation Models for Visual Artists' Creative Work
Kwon Ko, Gwanmo Park, Hyeon Jeon, Jaemin Jo, Juho Kim, Jinwook Seo
IUI 2023
paper / video / doi

Large-scale Text-to-image Generation Models (LTGMs) (e.g., DALL-E), self-supervised deep learning models trained on a huge dataset, have demonstrated the capacity for generating high-quality open-domain images from multi-modal input. Although they can even produce anthropomorphized versions of objects and animals, combine irrelevant concepts in reasonable ways, and give variation to any user-provided images, we witnessed such rapid technological advancement left many visual artists disoriented in leveraging LTGMs more actively in their creative works. Our goal in this work is to understand how visual artists would adopt LTGMs to support their creative works. To this end, we conducted an interview study as well as a systematic literature review of 72 system/application papers for a thorough examination. A total of 28 visual artists covering 35 distinct visual art domains acknowledged LTGMs’ versatile roles with high usability to support creative works in automating the creation process (i.e., automation), expanding their ideas (i.e., exploration), and facilitating or arbitrating in communication (i.e., mediation). We conclude by providing four design guidelines that future researchers can refer to in making intelligent user interfaces using LTGMs.

We-toon: A Communication Support System between Writers and Artists in Collaborative Webtoon Sketch Revision
Kwon Ko*, Subin An*, Gwanmo Park, Seung Kwon Kim, Daesik Kim, Bohyoung Kim, Jaemin Jo, Jinwook Seo
UIST 2022
paper / video / talk / doi

We present a communication support system, namely We-toon, that can bridge the webtoon writers and artists during sketch revision (i.e., character design and draft revision). In the highly iterative design process between the webtoon writers and artists, writers often have difficulties in precisely articulating their feedback on sketches owing to their lack of drawing proficiency. This drawback makes the writers rely on textual descriptions and reference images found using search engines, leading to indirect and inefficient communications. Inspired by a formative study, we designed We-toon to help writers revise webtoon sketches and effectively communicate with artists. Through a GAN-based image synthesis and manipulation, We-toon can interactively generate diverse reference images and synthesize them locally on any user-provided image. Our user study with 24 professional webtoon authors demonstrated that We-toon outperforms the traditional methods in terms of communication effectiveness and the writers’ satisfaction level related to the revised image.

Uniform Manifold Approximation with Two-phase Optimization
Hyeon Jeon*, Kwon Ko*, Soohyun Lee, Jaemin Jo, Jinwook Seo
VIS 2022 short
paper / code / video / talk / doi

We introduce Uniform Manifold Approximation with Two-phase Optimization (UMATO), a dimensionality reduction (DR) technique that improves UMAP to capture the global structure of high-dimensional data more accurately. In UMATO, optimization is divided into two phases so that the resulting embeddings can depict the global structure reliably while preserving the local structure with sufficient accuracy. As the first phase, hub points are identified and projected to construct a skeletal layout for the global structure. In the second phase, the remaining points are added to the embedding preserving the regional characteristics of local areas. Through quantitative experiments, we found that UMATO (1) outperformed widely used DR techniques in preserving the global structure while (2) producing competitive accuracy in representing the local structure. We also verified that UMATO is preferable in terms of robustness over diverse initialization methods, number of epochs, and subsampling techniques.

Distortion-Aware Brushing for Interactive Cluster Analysis in Multidimensional Projections
Hyeon Jeon, Michael Aupetit, Soohyun Lee, Kwon Ko, Youngtaek Kim, Jinwook Seo
Arxiv 2022 (Under review)
paper / code

Brushing is an everyday interaction in 2D scatterplots, which allows users to select and filter data points within a continuous, enclosed region and conduct further analysis on the points. However, such conventional brushing cannot be directly applied to Multidimensional Projections (MDP), as they hardly escape from False and Missing Neighbors distortions that make the relative positions of the points unreliable. To alleviate this problem, we introduce Distortion-aware brushing, a novel brushing technique for MDP. While users perform brushing, Distortion-aware brushing resolves distortions around currently brushed points by dynamically relocating points in the projection; the points whose data are close to the brushed data in the multidimensional (MD) space go near the corresponding brushed points in the projection, and the opposites move away. Hence, users can overcome distortions and readily extract out clustered data in the MD space using the technique. We demonstrate the effectiveness and applicability of Distortion-aware brushing through usage scenarios with two datasets. Finally, by conducting user studies with 30 participants, we verified that Distortion-aware brushing significantly outperforms previous brushing techniques in precisely separating clusters in the MD space, and works robustly regardless of the types or the amount of distortions in MDP.

Measuring and explaining the inter-cluster reliability of multidimensional projections
Hyeon Jeon, Kwon Ko, Jaemin Jo, Youngtaek Kim, Jinwook Seo
TVCG 2021 (Proc. VIS 2021)
paper / code / video / bibtex / doi / post

We propose Steadiness and Cohesiveness, two novel metrics to measure the inter-cluster reliability of multidimensional projection (MDP), specifically how well the inter-cluster structures are preserved between the original high-dimensional space and the low-dimensional projection space. Measuring inter-cluster reliability is crucial as it directly affects how well inter-cluster tasks (e.g., identifying cluster relationships in the original space from a projected view) can be conducted; however, despite the importance of inter-cluster tasks, we found that previous metrics, such as Trustworthiness and Continuity, fail to measure inter-cluster reliability. Our metrics consider two aspects of the inter-cluster reliability: Steadiness measures the extent to which clusters in the projected space form clusters in the original space, and Cohesiveness measures the opposite. They extract random clusters with arbitrary shapes and positions in one space and evaluate how much the clusters are stretched or dispersed in the other space. Furthermore, our metrics can quantify pointwise distortions, allowing for the visualization of inter-cluster reliability in a projection, which we call a reliability map. Through quantitative experiments, we verify that our metrics precisely capture the distortions that harm inter-cluster reliability while previous metrics have difficulty capturing the distortions. A case study also demonstrates that our metrics and the reliability map 1) support users in selecting the proper projection techniques or hyperparameters and 2) prevent misinterpretation while performing inter-cluster tasks, thus allow an adequate identification of inter-cluster structure.

Mixed-Initiative Approach to Extract Data from Pictures of Medical Invoice
Seokweon Jung, Kiroong Choe, Seokhyeon Park, Kwon Ko, Youngtaek Kim, Jinwook Seo
PacificVis 2021 short
paper / video / bibtex / doi

Extracting data from pictures of medical records is a common task in the insurance industry as the patients often send their medical invoices taken by smartphone cameras. However, the overall process is still challenging to be fully automated because of low image quality and variation of templates that exist in the status quo. In this paper, we propose a mixed-initiative pipeline for extracting data from pictures of medical invoices, where deep-learning-based automatic prediction models and task-specific heuristics work together under the mediation of a user. In the user study with 12 participants, we confirmed our mixed-initiative approach can supplement the drawbacks of a fully automated approach within an acceptable completion time. We further discuss the findings, limitations, and future works for designing a mixed-initiative system to extract data from pictures of a complicated table.

Progressive Uniform Manifold Approximation and Projection
Kwon Ko, Jaemin Jo, Jinwook Seo
EuroVis 2020 short
paper / code / talk / bibtex / doi

We present a progressive algorithm for the Uniform Manifold Approximation and Projection (UMAP), called the Progressive UMAP. Based on the theory of Riemannian geometry and algebraic topology, UMAP is an emerging dimensionality reduction technique that offers better versatility and stability than t-SNE. Although UMAP is also more efficient than t-SNE, it still suffers from an initial delay of a few minutes to produce the first projection, which limits its use in interactive data exploration. To tackle this problem, we improve the sequential computations in UMAP by making them progressive, which allows people to incrementally append a batch of data points into the projection at the desired pace. In our experiment with the Fashion MNIST dataset, we found that Progressive UMAP could generate the first approximate projection within a few seconds while also sufficiently capturing the important structures of the high-dimensional dataset.

PROJECTS

Abstract Sketch to Character
Spring 2022

Convert human-drawn abstract sketch image to an anime character with varying styles

Playing Atari with DQN
Winter 2019
code / video

Implemented DQN algorithm (Mnih et al. 2013) that can play one of the Atari games, 'Pong', and can beat the opponent

Musical Structure Visualization with MIDI Data
Fall 2019
demo / code / video

Information Visualization project at SNU

Korean Web Novel Generation
Spring 2018 - Summer 2018
paper / code / wiki / link / media

Made deep learning-based Korean sentence recommender. Received 20,000,000 KRW of prize money.

MISC

I love playing piano. Also, I like swimming a lot, so I go swimming almost everyday. I am a big fan of Tottenham Hotspur since 2009. I own more than 400 comic books and like reading Murakami Haruki's short novels. I love this quote the most:

Dance like nobody's watching; Love like you've never been hurt; Work like you don't need money; Live like it's heaven on earth.

30-Minute Chat with Me I am open to discussing research ideas, exploring collaborations, offering technical advice, or supporting startups and tech-focused individuals. I am also happy to assist with career guidance, PhD applications, or general advice on graduate life. Click below to book a 30-minute chat. I will do my best to help.

           

Referred to existing websites such as this and this.