{"id":447,"date":"2021-09-17T00:00:00","date_gmt":"2021-09-17T00:00:00","guid":{"rendered":"https:\/\/en.pingcap.com\/blog\/get-started-with-juicefs-using-tikv\/"},"modified":"2024-08-20T06:38:36","modified_gmt":"2024-08-20T13:38:36","slug":"get-started-with-juicefs-using-tikv","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/get-started-with-juicefs-using-tikv\/","title":{"rendered":"Getting Started with JuiceFS Using TiKV"},"content":{"rendered":"

As a cloud-native distributed storage system, JuiceFS was designed into a plug-in structure at the beginning of its birth to ensure that new technologies can be continuously integrated into the JuiceFS ecosystem. According to your needs, you can flexibly choose two core components, the data storage engine and the metadata engine, according to their needs.<\/p>\n

The data storage engine is mainly object storage. It supports almost all public and private cloud object storage services, as well as KV storage, WebDAV, and local disks. The metadata engine supports databases such as Redis, MySQL, PostgreSQL, and SQLite.<\/p>\n

The newly released JuiceFS v0.16 officially supports TiKV, a distributed key-value database, which further meets the requirements for elastic scaling in high-performance, large-scale data storage.<\/p>\n

In this article, I’ll share with you how to use JuiceFS and select TiKV as a metadata engine.<\/p>\n

<\/span>TiKV, a highly scalable KV database<\/span><\/h2>\n

TiKV is a distributed transactional key-value database featuring high scalability, low latency, and ease of use. It easily processes over a trillion rows of data in petabytes.<\/p>\n

TiKV supports unlimited horizontal scaling and provides a distributed transaction interface that meets ACID compliance. It also ensures data consistency and high availability of multiple replicas using the Raft protocol.<\/p>\n

\"juicefs-tikv-architecture\"

TiKV architecture<\/p><\/div>\n

TiKV was developed by PingCAP and is a graduated project of the Cloud Native Computing Foundation (CNCF).<\/p>\n

<\/span>Install TiKV<\/span><\/h2>\n

PingCAP provides TiUP, a package manager, which can easily install TiKV on Linux or macOS.<\/p>\n

Install TiUP<\/h3>\n

This command downloads and installs the appropriate TiUP version to your system.<\/p>\n

curl --proto '=https' --tlsv1.2 -sSf https:\/\/tiup-mirrors.pingcap.com\/install.sh | sh\n<\/code><\/pre>\n
For the settings to take effect, you can open a new terminal, or manually declare the global environment variable as follows:<\/p>\n
source .bash_profile\n<\/code><\/pre>\n
Now try to execute the command tiup -v<\/code> and see the version information similar to the following, which means the installation is successful.<\/p>\n
tiup -v\n\n1.5.4 tiup\nGo Version: go1.16.6\nGit Ref: v1.5.4\nGitHash: b629670276269cd1518eb28f362a5180135cc985\n<\/code><\/pre>\n
Deploy a TiKV cluster<\/h4>\n
For testing purposes, we’ll use the TiUP playground<\/code> component to install a minimal TiKV test cluster in the local environment.<\/p>\n
tiup playground --mode tikv-slim\n<\/code><\/pre>\n
After the deployment is successful, the terminal displays a message similar to the following:<\/p>\n
PD client endpoints: [127.0.0.1:2379]\nTo view the Prometheus:[ http:\/\/127.0.0.1:9090](http:\/\/127.0.0.1:9090\/)\nTo view the Grafana:[ ](http:\/\/127.0.0.1:3000\/)http:\/\/127.0.0.1:3000\n<\/code><\/pre>\n
Among them, <127.0.0.1:2379><\/code> is the Placement Driver<\/a> (PD) address, which manages and schedules the TiKV cluster. JuiceFS will interact with TiKV through this address. The other two addresses are Prometheus and Grafana services, which are used for monitoring and data visualization of TiKV clusters.<\/p>\n
Note<\/strong>: The playground component of TiUP is mainly used to quickly build a minimal test cluster of TiDB and TiKV in the local environment. For production environment deployment, please refer to TiKV Official Document<\/a>.<\/p>\n
<\/span>Install JuiceFS<\/span><\/h2>\n
JuiceFS supports Linux, Windows, and macOS systems. You only need to download the corresponding version of the client program and place it in the executable path of the system. For example, I use a Linux distribution, so I can install the latest version of the client by executing the following commands in sequence.<\/p>\n
Check the current system information and set temporary environment variables:<\/p>\n
JFS_LATEST_TAG=$(curl -s https:\/\/api.github.com\/repos\/juicedata\/juicefs\/releases\/latest | grep 'tag_name' | cut -d '\"' -f 4 | tr -d 'v')\n<\/code><\/pre>\n
Download the latest version of the JuiceFS client corresponding to your system:<\/p>\n
wget \"https:\/\/github.com\/juicedata\/juicefs\/releases\/download\/v${JFS_LATEST_TAG}\/juicefs-${JFS_LATEST_TAG}-linux-amd64.tar.gz\"\n<\/code><\/pre>\n
Unzip the package:<\/p>\n
mkdir juice && tar -zxvf \"juicefs-${JFS_LATEST_TAG}-linux-amd64.tar.gz\" -C juice\n<\/code><\/pre>\n
Install JuiceFS client to \/usr\/local\/bin<\/code>:<\/p>\n
sudo install juice\/juicefs \/usr\/local\/bin\n<\/code><\/pre>\n
Execute the command and see the help information returned, which means that the client is installed successfully.<\/p>\n
juicefs\n\nNAME:\n  juicefs - A POSIX file system built on Redis and object storage.\n\nUSAGE:\n  juicefs [global options] command [command options] [arguments...]\n\nVERSION:\n  0.16.1 (2021-08-16 2edcfc0)\n\nCOMMANDS:\n  format   format a volume\n  mount    mount a volume\n  umount   unmount a volume\n  gateway  S3-compatible gateway\n  sync     sync between two storage\n  rmr      remove directories recursively\n  info     show internal information for paths or inodes\n  bench    run benchmark to read\/write\/stat big\/small files\n  gc       collect any leaked objects\n  fsck     Check consistency of file system\n  profile  analyze access log\n  stats    show runtime stats\n  status   show status of JuiceFS\n  warmup   build cache for target directories\/files\n  dump     dump metadata into a JSON file\n  load     load metadata from a previously dumped JSON file\n  help, h  Shows a list of commands or help for one command\n\nGLOBAL OPTIONS:\n  --verbose, --debug, -v  enable debug log (default: false)\n  --quiet, -q             only warning and errors (default: false)\n  --trace                 enable trace log (default: false)\n  --no-agent              Disable pprof (:6060) and gops (:6070) agent (default: false)\n  --help, -h              show help (default: false)\n  --version, -V           print only the version (default: false)\n\nCOPYRIGHT:\n  AGPLv3\n<\/code><\/pre>\n
You can also visit the JuiceFS GitHub Releases<\/a> page to select other versions for manual installation.<\/p>\n
<\/span>Use TiKV in JuiceFS<\/span><\/h2>\n
Here I refer to JuiceFS Quick Start Guide<\/a>, and build a MinIO object storage locally. The access address is http:\/\/127.0.0.1:9000, and Access Key ID<\/code> and Access Key Secret<\/code> are both minioadmin<\/code>.<\/p>\n
Create a file system<\/h3>\n
The following command uses the format<\/code> subcommand provided by the JuiceFS client to create a file system named mystor<\/code>. In this command, TiKV database address format uses the PD address of the TiKV cluster:<\/p>\n
juicefs format \n    --storage minio \n    --bucket[ http:\/\/127.0.0.1:9000\/mystor](http:\/\/127.0.0.1:9000\/mystor) \n    --access-key minioadmin \n    --secret-key minioadmin \n    tikv:\/\/127.0.0.1:2379\/mystor \n    mystor\n<\/code><\/pre>\n
Parameter Description:<\/p>\n