{"id":119,"date":"2016-12-19T00:00:00","date_gmt":"2016-12-19T00:00:00","guid":{"rendered":"https:\/\/en.pingcap.com\/blog\/adding-built-in-function\/"},"modified":"2024-05-23T19:51:15","modified_gmt":"2024-05-24T02:51:15","slug":"adding-built-in-function","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/adding-built-in-function\/","title":{"rendered":"Adding Built-in Functions"},"content":{"rendered":"<p>This document describes how to add built-in functions to TiDB.<\/p>\n<h3>Background<\/h3>\n<p>How is the SQL statement executed in TiDB?<\/p>\n<p>The SQL statement is parsed to an abstract syntax tree (AST) by the parser first and then uses Query Optimizer to generate an execution plan. The plan can then be executed to get the result. This process involves how to access the data in the table, and how to filter, calculate, sort, aggregate, and distinct the data, etc. For a built-in function, the most important part is to parse and to evaluate.<\/p>\n<p>For parsing, it is redundant work because you should know how to write YACC commands and how to modify TiDB syntax parser. But we have finished this work for you and syntax parsing of most built-in functions is done.<\/p>\n<p>As for evaluation, it should be finished in the TiDB expression evaluation framework. Each built-in function is considered as an expression indicated by <code>ScalarFunction<\/code> and obtains the corresponding function type and function signature through the function name and parameters to evaluate.<\/p>\n<p>The procedure discussed above is complicated for users who are not familiar with TiDB. We have finished syntax parsing and function signature confirmation of most unimplemented functions. But implementation is left empty. In other words, locating and completing the empty implementation makes a Pull Request (PR).<\/p>\n<h3>The procedure to add a built-in function<\/h3>\n<p>The following procedure describes how to add a built-in function.<\/p>\n<ol>\n<li>Locate the unimplemented function.\n<ol>\n<li>Search for <code>errFunctionNotExists<\/code> in the <code>expression<\/code> directory of TiDB source code. You can find all the unimplemented functions.<\/li>\n<li>Choose a function you are interested in. Take the SHA2 function as an example:\n<pre><code>func (b *builtinSHA2Sig) eval(row []types.Datum) (d types.Datum, err error) {\nreturn d, errFunctionNotExists.GenByArgs(\"SHA2\")\n}\n<\/code><\/pre>\n<\/li>\n<\/ol>\n<\/li>\n<li>Implement the function signature.This step is to implement <code>eval<\/code>. For the function features, see MySQL documentation. For the specific implementation method, see the method of implemented functions.<\/li>\n<li>Add the type inference information to the <code>typeinferer<\/code> file.Add the type of the returned result of the function to <code>handleFuncCallExpr()<\/code> in the the <code>plan\/typeinferer.go<\/code> file and make sure the result is consistent with the result in MySQL. See <a href=\"https:\/\/github.com\/pingcap\/tidb\/blob\/ee6da7aeeeadf8b29a38b988d3b17aa585aea327\/structure\/type.go\">MySQL Const<\/a> for the complete list of the type definition.\n<p><strong>Note:<\/strong> For most functions, you need to input the type of the returned result and obtain the length of the returned result.<\/li>\n<li>Add a unit test case.Add a unit test case for the function to the <code>expression<\/code> directory. Add a unit test case of <code>typeinferer<\/code> to the <code>plan\/typeinferer_test.go<\/code> file.<\/li>\n<li>Run the <code>make dev<\/code> command and make sure all the test cases can pass.<\/li>\n<\/ol>\n<div class=\"trackable-btns\"><a href=\"\/download\"><button>Download TiDB<\/button><\/a><br \/>\n<a href=\"https:\/\/share.hsforms.com\/1e2W03wLJQQKPd1d9rCbj_Q2npzm\"><button>Subscribe to Blog<\/button><\/a><\/div>\n<h3>Example<\/h3>\n<p>Take the <a href=\"https:\/\/github.com\/pingcap\/tidb\/pull\/2781\/files\">Pull Request<\/a> to add the <code>SHA1()<\/code> function as an example:<\/p>\n<ol>\n<li>Open the <code>expression\/builtin_encryption.go<\/code> file and complete the evaluation of <code>SHA1()<\/code>.\n<pre><code>func (b *builtinSHA1Sig) eval(row []types.Datum) (d types.Datum, err error) {\n    \/\/ Evaluate the arguments. In most cases, you do not need to make any modification.\n    args, err := b.evalArgs(row)\n    if err != nil {\n        return types.Datum{}, errors.Trace(err)\n    }\n    \/\/ See MySQL documentation for the meaning of each argument.\n    \/\/ SHA\/SHA1 function only accept 1 parameter\n    arg := args[0]\n    if arg.IsNull() {\n        return d, nil\n    }\n    \/\/ The type of the argument value is changed. See \"util\/types\/datum.go\" for the function implementation.\n    bin, err := arg.ToBytes()\n    if err != nil {\n        return d, errors.Trace(err)\n    }\n    hasher := sha1.New()\n    hasher.Write(bin)\n    data := fmt.Sprintf(\"%x\", hasher.Sum(nil))\n    \/\/ Set the return value.\n    d.SetString(data)\n    return d, nil\n}\n<\/code><\/pre>\n<\/li>\n<li>Add a unit test case for the function implementation. See <code>expression\/builtin_encryption_test.go<\/code>:\n<pre><code>var shaCases = []struct {\n    origin interface{}\n    crypt  string\n }{\n    {\"test\", \"a94a8fe5ccb19ba61c4c0873d391e987982fbbd3\"},\n    {\"c4pt0r\", \"034923dcabf099fc4c8917c0ab91ffcd4c2578a6\"},\n    {\"pingcap\", \"73bf9ef43a44f42e2ea2894d62f0917af149a006\"},\n    {\"foobar\", \"8843d7f92416211de9ebb963ff4ce28125932878\"},\n    {1024, \"128351137a9c47206c4507dcf2e6fbeeca3a9079\"},\n    {123.45, \"22f8b438ad7e89300b51d88684f3f0b9fa1d7a32\"},\n }\n\n func (s *testEvaluatorSuite) TestShaEncrypt(c *C) {\n    defer testleak.AfterTest(c)() \/\/ The tool for monitoring goroutine leak. You can just copy it.\n    fc := funcs[ast.SHA]\n    for _, test := range shaCases {\n        in := types.NewDatum(test.origin)\n        f, _ := fc.getFunction(datumsToConstants([]types.Datum{in}), s.ctx)\n        crypt, err := f.eval(nil)\n        c.Assert(err, IsNil)\n        res, err := crypt.ToString()\n        c.Assert(err, IsNil)\n        c.Assert(res, Equals, test.crypt)\n    }\n    \/\/ test NULL input for sha\n    var argNull types.Datum\n    f, _ := fc.getFunction(datumsToConstants([]types.Datum{argNull}), s.ctx)\n    crypt, err := f.eval(nil)\n    c.Assert(err, IsNil)\n    c.Assert(crypt.IsNull(), IsTrue)\n}\n<\/code><\/pre>\n<blockquote><p><strong>Note:<\/strong> Besides conventional cases, you had better add some exceptional cases in which, for example, the input value is &#8220;nil&#8221; or the arguments of various types.<\/p><\/blockquote>\n<\/li>\n<li>Add the type inference information and the test case. See <code>plan\/typeinferer.go<\/code> and <code>plan\/typeinferer_test.go<\/code>:\n<pre><code>case ast.SHA, ast.SHA1:\n    tp = types.NewFieldType(mysql.TypeVarString)\n    chs = v.defaultCharset\n    tp.Flen = 40\n<\/code><\/pre>\n<pre><code>    {`sha1(123)`, mysql.TypeVarString, \"utf8\"},\n    {`sha(123)`, mysql.TypeVarString, \"utf8\"},\n<\/code><\/pre>\n<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>TiDB code is updated and the procedure of adding built-in functions is greatly simplified. This document describes how to add built-in functions to TiDB.<\/p>","protected":false},"author":8,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ub_ctt_via":"","footnotes":""},"categories":[18],"tags":[29],"class_list":["post-119","post","type-post","status-publish","format-standard","hentry","category-community","tag-tutorial"],"acf":[],"featured_image_src":null,"author_info":{"display_name":"TiDB Team","author_link":"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Adding Built-in Functions | TiDB<\/title>\n<meta name=\"description\" content=\"Learn about the benefits of TiDB and the TiDB Cloud solutions from PingCAP. Read our latest post &quot;Adding Built-in Functions&quot; here.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/adding-built-in-function\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Adding Built-in Functions | TiDB\" \/>\n<meta property=\"og:description\" content=\"Learn about the benefits of TiDB and the TiDB Cloud solutions from PingCAP. Read our latest post &quot;Adding Built-in Functions&quot; here.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/blog\/adding-built-in-function\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:published_time\" content=\"2016-12-19T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-24T02:51:15+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2024\/09\/11005522\/Homepage-Ad.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"714\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"TiDB Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"\uae00\uc4f4\uc774\" \/>\n\t<meta name=\"twitter:data1\" content=\"TiDB Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04\" \/>\n\t<meta name=\"twitter:data2\" content=\"4\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/\"},\"author\":{\"name\":\"TiDB Team\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74\"},\"headline\":\"Adding Built-in Functions\",\"datePublished\":\"2016-12-19T00:00:00+00:00\",\"dateModified\":\"2024-05-24T02:51:15+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/\"},\"wordCount\":474,\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"keywords\":[\"Tutorial\"],\"articleSection\":[\"Community\"],\"inLanguage\":\"ko-KR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/\",\"url\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/\",\"name\":\"Adding Built-in Functions | TiDB\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"datePublished\":\"2016-12-19T00:00:00+00:00\",\"dateModified\":\"2024-05-24T02:51:15+00:00\",\"description\":\"Learn about the benefits of TiDB and the TiDB Cloud solutions from PingCAP. Read our latest post \\\"Adding Built-in Functions\\\" here.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Adding Built-in Functions\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74\",\"name\":\"TiDB Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"caption\":\"TiDB Team\"},\"url\":\"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Adding Built-in Functions | TiDB","description":"Learn about the benefits of TiDB and the TiDB Cloud solutions from PingCAP. Read our latest post \"Adding Built-in Functions\" here.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pingcap.com\/ko\/blog\/adding-built-in-function\/","og_locale":"ko_KR","og_type":"article","og_title":"Adding Built-in Functions | TiDB","og_description":"Learn about the benefits of TiDB and the TiDB Cloud solutions from PingCAP. Read our latest post \"Adding Built-in Functions\" here.","og_url":"https:\/\/www.pingcap.com\/ko\/blog\/adding-built-in-function\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_published_time":"2016-12-19T00:00:00+00:00","article_modified_time":"2024-05-24T02:51:15+00:00","og_image":[{"width":1440,"height":714,"url":"https:\/\/static.pingcap.com\/files\/2024\/09\/11005522\/Homepage-Ad.png","type":"image\/png"}],"author":"TiDB Team","twitter_card":"summary_large_image","twitter_creator":"@PingCAP","twitter_site":"@PingCAP","twitter_misc":{"\uae00\uc4f4\uc774":"TiDB Team","\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04":"4\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/#article","isPartOf":{"@id":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/"},"author":{"name":"TiDB Team","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74"},"headline":"Adding Built-in Functions","datePublished":"2016-12-19T00:00:00+00:00","dateModified":"2024-05-24T02:51:15+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/"},"wordCount":474,"publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"keywords":["Tutorial"],"articleSection":["Community"],"inLanguage":"ko-KR"},{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/","url":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/","name":"Adding Built-in Functions | TiDB","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"datePublished":"2016-12-19T00:00:00+00:00","dateModified":"2024-05-24T02:51:15+00:00","description":"Learn about the benefits of TiDB and the TiDB Cloud solutions from PingCAP. Read our latest post \"Adding Built-in Functions\" here.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/blog\/adding-built-in-function\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"Adding Built-in Functions"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]},{"@type":"Person","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74","name":"TiDB Team","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/","url":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","caption":"TiDB Team"},"url":"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/"}]}},"grav_blocks":false,"card_markup":"<a class=\"card-resource bg-white\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/adding-built-in-function\/\"><div class=\"card-resource__content-container\"><div class=\"card-resource__content-head\"><div class=\"card-resource__category\">Community<\/div><\/div><h5 class=\"card-resource__title\">Adding Built-in Functions<\/h5><\/div><\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/119","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/comments?post=119"}],"version-history":[{"count":3,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/119\/revisions"}],"predecessor-version":[{"id":17164,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/119\/revisions\/17164"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=119"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/categories?post=119"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/tags?post=119"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}