Virtualization Methods for Securing Online Exam
Sandbox refers to a controlled environment for code execution. It imposes artificial limits on processes running on top of it. These limits may include limits to file system access, execution time limit, blocked access to other processes, and/or memory/disk usage limit. Sandbox has been widely used for various purposes, ranging from the auto-grader for programming contests  to web browser isolation . The idea of securing online exam remotely has been implemented by various research [6, 12] and software packages [5, 2, 3]. Jung et. al. presents a system for securing remote exam without proctor supervision, using group cryptography and remote monitoring of various inputs , while  proposes various control procedures for securing exam and the use of custom software packages. SWITCH  presents SafeExamBrowser (SEB), a shell sandbox for the online exam. Know More When running, it locks the desktop, defines a list of allowed processes that can be run during the exam, defmes a list of white listed and blacklisted IPs for the exam, and blocks several functionalities of the OS, such as task manager, keyboard shortcuts, context menu (right click), log out, system services, locking the workstation, and exiting the app itself. Moreover, [2, 3] provides a proctoring-as-a-service model, which, in addition to providing software packages similar to , each exam taker is also remotely monitored by a human proctor through webcam and microphone. Virtualization is a method that defmes a set of interfaces that maps a system/a subsystem onto the interfaces and resources of a real system, which implements these virtualized interfaces. Formally, it involves the construction of an isomorphism between a guest system and a host . Let S1. and Sj. represents states of a system in the guest system, while S[ and Sj represents states of a system in the host system. For every state S, there is a virtualization function V that maps S to 51. As a corollary, for function ,9, a sequence of operations that maps S[ to Sj in the guest, there is a corresponding sequence of operations ‘ll that maps S[ to SJ. Virtualization is used for various purposes, such as testing/running binaries from another OS or for creating a sandbox . It should be highlighted, however, that virtualization is not always used for sandboxing purposes, and vice versa. The virtualization methods explored in this research are hardware level virtualization and operating system level virtualization. Hardware level virtualization is defined as emulating a set of hardware, usually for running a full-fledged, sandboxed operating system, through a hypervisor. A hypervisor, software that manages such virtual machines, emulates various elements of the hardware, such as a processor, memory, hard disk, and network cards. Hardware system calls from the guest operating system are intercepted by the hypervisor. This allows applications on the guest system to be run on a different kernel than the host. Based on which it is installed, hypervisor implementations are categorized to Type-1 and Type-2. Type-l hypervisor runs directly on top of the hardware, whereas Type-2 hypervisor runs on top of the host OS. Fig. 1 illustrates the differences between the two types of hypervisors. Securing online exam through hardware level virtualization method has been explored by Reuter et. al.. In , Reuter et. a!. describes the general concept for online exams using VDI and SEB, including process descriptions for planning and conducting such exams . It should be highlighted that the exam in  is conducted in a managed, local environment, that is equipped with a high bandwidth, redundant network. Such a network setup is common for VDI since a large amount of data – especially image/graphics data from and to the virtual desktops -is transferred across the network. Operating system (OS) level virtualization is a virtualization layer implemented on top of the operating system. It is achieved through isolating running applications in a separate, isolated, user space instances, which is more commonly known as containers . Compared to hardware level virtualization, OS level virtualization incurs lower perfonnance overhead. All applications running on the container uses the same system calls provided by the host system, whereas both types of hardware virtualization emulate the whole system – both the kernel and the system calls. As a consequence, OS-level virtualization is only capable of running applications that run on the same kernel as the host OS.  provides measurements comparing the perfonnance of hardware level virtualization and OS level virtualization. OS level virtualization in Windows has been explored in various research [15, 16, 14]. In , Allayes presents a method of creating a lightweight virtual machine for the purpose of creating a portable application, i.e. application that preserves its settings when executed in different clients, via user-mode hooks. Moreover, Yu et. al.  describes a method for creating a lightweight virtual machine, called FVM. FVM uses Detours library , implemented as a Windows minifilter driver at the kernel space, that intercepts particular system calls and redirects them. It covers isolation and control of various operating system resources, which include file system isolation, registry isolation, object isolation (mutex, semaphore, event, timer, ports, etc.), network interface isolation, interprocess communication isolation, and daemon service isolation. Finally, Porter et. al.  presents a method of creating a fixed set of abstractions over Windows to construct a library OS, that encapsulates application services over a fixed set of abstractions, i.e. hardware services and user services. The library OS prototype aims for security, host independence, and ease migration of processes across different computers. By isolating the process and serving it over Remote Desktop Protocol (RDP), it achieves its isolation purposes. Hooking is a technique that can be used to implement isolation of user space instances. By hooking, or also known as runtime code patching, a programmer may alter the behavior of a particular operating system or program, by intercepting function calls of the application and redirect it to a custom function. One of the ways of achieving this during runtime is to replace the first few code instructions of the runtime function to jump to the injected code. This technique is possible on an operating system that allows dynamic modules to be loaded by applications during runtime. Examples of libraries that implement this technique are Detours , the original library, and EasyHook , which expands upon the techniques of .
EXPERIMENTS AND PROOF OF CONCEPT
Based on the use cases illustrated in Fig. 2, we created seven test scenarios to test our implementation of the abuse cases. These test scenarios are used both by the hardware level implementation and the OS level implementation (one using ntdll. dll hooks, another one using kerne132. dll hooks). In total, there are 25 tests run on each of virtualization implementation, both covering the use cases (18 test cases) and the abuse cases (7 test cases). Any error or application crash is treated as a failure. A. Test Environment Our hardware level virtualization implementation uses a customized Arch Linux live image, run on top of VirtualBox. In the live image, applications used in our online exam are bundled. These include Geany, an IDE built on GTK+2 runtime, Mozilla Firefox a web browser, MATE Terminal, the command line shell for MATE Desktop Environment, and Caja, the graphical file manager. Moreover, we also bundled gee and JDK, as compilers for C/C++ and Java, respectively, inside the image. On the other hand, our OS level virtualization implementation is run directly on top of Windows. Similar to our hardware level implementation, inside it, applications used in our online exam are bundled, i.e. Geany and Mozilla Firefox. However, since Windows already has its graphical file manager (Windows Explorer) and command line shell (Command Prompt), these two are used instead. Moreover, since our current implementations of the hooks are limited (i.e. an application cannot be hooked since it was created; it can only be hooked when it is running), we do not use installed compilers, since their running time is short (around 1-5 seconds), especially for compiling programs used for online exams (these programs consists of less than 1000 line of codes). Currently, both of our OS level implementation only logs access to the filesystem. No further action is done by them. B. Use case tests This section describes the tests for implementation of hardware level virtualization and OS level virtualization. All of these tests are taken from the normal scenarios of the use cases illustrated in Fig. 2. 1) Hardware level virtualization All test cases are passed by our implementation. The client can use the virtual machine as it is. All of the tasks, including programming, compiling, testing program, accessing the exam
website, and uploading/downloading files, can normally be done. Therefore, the hardware level virtualization passes all of our 18 test cases. 2) OS level virtualization Since our current implementation only logs access to the filesystem, there are no differences in the functions of our tested applications. All of our applications can run nonnally. Also, all accesses to the file system are recorded by both of the implementation of our applications. No crashes or errors occur by using these applications. A different behavior is shown by our as level implementations. The kerne132. dll hooks only detect accesses to file system made by Geany and Mozilla Firefox. Accesses to file system made by Command Prompt and Windows Explorer are not detected. Only the ntdll. dll hooks successfully detect all file system accesses made by all applications, including Command Prompt and Windows Explorer. Since it fails to detect such access, the kerne132 . dll hooks fail on all test cases of Command Prompt and Windows Explorer, while the ntdll. dll hooks pass all test cases. C. Abuse case tests This section describes the tests of the implementation of hardware level virtualization and as level virtualization. These tests are taken from scenarios of abuse cases in Fig. 2. 1) Hardware level virtualization SEB blocks access to the host OS’s shell. Therefore, access to files of the host as is also blocked. So far, we have not found a possible way for a regular exam taker to get out of the sandbox. In addition, network access is blocked through the configuration of the VM. Moreover, an exam taker cannot exit SEB if he/she does not have an exit password. This prevents him/her to access files on the host as during an exam. In summary, all abuse test cases are passed. 2) OS level virtualization As mentioned at Section B.2, only ntdll. dll hooks can capture file system accesses made by all applications. However, since only monitoring is done, no actual protection can be applied during the application runtime. Therefore, the users may still cheat during exams through the bundled applications. Cheats can only be detected post-exams. In addition, file system hooks cannot detect any access to websites other than the exam website. In summary, both of the as level virtualization implementations fail on all of the four test cases. Fig. 4 illustrates 25 test cases that cover the number of minimum test cases derived from the use case/abuse case diagram in Fig. 2 (9 cases) and extra cases (16 test cases). Test cases from the abuse cases in Fig 2 are related to the prevention of fraud/cheating during an online exam. The number of minimum test cases from Fig. 2 is calculated by counting the number of terminal use cases/abuse cases. Extra test cases are due to different applications being used for testing (e.g. for OS-level implementation, Create File use case is tested using Geany, Windows Explorer, and MATE Terminal).
RESULTS AND DISCUSSION
A. Results Based on our experiments, hardware-level virtualization passes all test cases. It should be highlighted, however, that the tests conducted may not reflect all possible ways of detecting cheating on exams, since cheating, in general, may involve methods beyond of the test cases used to test the abuse cases on IV.C (e.g. using malicious program that can circumvent the security before the exam). For the as level virtualization, kerne132. dll hooks fail to detect accesses to the file system made by Command Prompt and Windows Explorer, while ntdll. dll hooks successfully detect these accesses. One possible explanation is that these programs do not reside on the Win32 subsystem since this subsystem is commonly used by third-party applications. B. Comparing the virtualization methods From the level of isolation provided, hardware level virtualization provides better isolation compared to as level virtualization. The hypervisor executes a separate kernel, on which isolated applications run on top of it. Since by default the guest as and host as are isolated, no communication between these two OSes can occur. From the ease of implementation aspect, hardware level virtualization is better than as level virtualization. Since the hypervisor used during the test, VirtualBox, can emulate various x86 instructions, it is easier to use operating systems that support x86 (e.g. Windows, Linux, *BSD). This enables the examiner in SEEI ITB to match the virtual image with the computers currently in use during exams. Moreover, VirtualBox is also cross-platform. This enables it to be used in different OSes. In addition, VirtualBox is stable and also provides its users with well-written documents and flexibility of configurations. This is contrasted by as level virtualization, which is OS-specific. A different implementation of this type of virtualization requires a different analysis of the file systemrelated system calls used by the as. Furthermore, as of this writing, the library used for implementing as level virtualization, EasyHook, is still in beta version. Lack of documentation and stability issues may arise when developing using this library. Besides, not all of Windows internal functions, especially functions of ntdll. dll, are welldocumented. Therefore, from our experience, OS-level virtualization requires more effort in implementation compared to hardware level virtualization. In addition, it should be highlighted that usage of SEB also aids in isolating the hypervisor from user-made customizations. SEB adds a protection to VirtualBox, since by default, VirtualBox allows customization of the guest VM. Since SEB is not available on Linux (as of this writing, it is available only on Windows and OS X), a similar application should be used when developing an exam client sandbox for Linux. From the performance aspect, OS level virtualization gains an advantage over hardware level virtualization. As presented by [7, 16], OS-level virtualization causes less overhead compared to the hardware level. This may be beneficial for online exams that use low-end hardware. However, performance gain may result in reduced security, since from a conceptual level, OS level virtualization only isolates the user spaces of the applications. It cannot detect any cheating attempts that use a modification of the host OS’s kernel. Even though hardware level virtualization is not fully safe from the same cheating method, such method requires more effort to be implemented for the hardware level virtualization, since hardware level virtualization uses a separate kernel to execute the applications.