Popular Releases
Popular Libraries
New Libraries
Top Authors
Trending Kits
Trending Discussions
Learning
v2.1.7 | |
四月更新 Pro Max | |
Bilibili Helper v1.2.30 | |
BILIBILI-HELPER-v1.0.3 |
Bilibili-Evolved v2.1.7 |
DisableWinTracking |
Bili.Uwp 四月更新 Pro Max |
bilibili-helper-o Bilibili Helper v1.2.30 |
BILIBILI-HELPER BILIBILI-HELPER-v1.0.3 |
1
5 Libraries
481
2
5 Libraries
1095
3
5 Libraries
1187
4
4 Libraries
43
5
4 Libraries
1023
6
4 Libraries
99
7
4 Libraries
236
8
4 Libraries
493
9
4 Libraries
154
10
3 Libraries
145
1
5 Libraries
481
2
5 Libraries
1095
3
5 Libraries
1187
4
4 Libraries
43
5
4 Libraries
1023
6
4 Libraries
99
7
4 Libraries
236
8
4 Libraries
493
9
4 Libraries
154
10
3 Libraries
145
No Trending Kits are available at this moment for Bilibili
QUESTION
How to insert JSON containing escape codes into a JSONB column in PostgreSQL using GORM
Asked 2021-Jul-08 at 18:16I'm trying to store the JSON bytes to PostgreSQL, but there's a problem.
\u0000 cannot be converted to text.
As you can see below, the JSON contains escape sequences such as \u0000
, which it seems PostgreSQL is interpreting as unicode characters, not JSON strings.
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12
Origin API data can be access form Here. There is \u0000 in it:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106
The struct for save is:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106type RawJSONData struct {
107 ID uint64 `gorm:"primarykey" json:"id"`
108 CreatedAt time.Time `json:"-"`
109 DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
110 Data datatypes.JSON `json:"data"`
111 URL string `gorm:"index" json:"url"`
112}
113
datatypes.JSON
is from gorm.io/datatypes
. It seems just is json.RawMessage
, it is (extend from?) a []byte
.
I use PostgreSQL's JSONB
type for storage this data.
Table:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106type RawJSONData struct {
107 ID uint64 `gorm:"primarykey" json:"id"`
108 CreatedAt time.Time `json:"-"`
109 DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
110 Data datatypes.JSON `json:"data"`
111 URL string `gorm:"index" json:"url"`
112}
113create table raw_json_data
114(
115 id bigserial not null constraint raw_json_data_pke primary key,
116 created_at timestamp with time zone,
117 deleted_at timestamp with time zone,
118 data jsonb,
119 url text
120);
121
ANSWER
Answered 2021-Jul-08 at 16:05Well, I solved it by:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106type RawJSONData struct {
107 ID uint64 `gorm:"primarykey" json:"id"`
108 CreatedAt time.Time `json:"-"`
109 DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
110 Data datatypes.JSON `json:"data"`
111 URL string `gorm:"index" json:"url"`
112}
113create table raw_json_data
114(
115 id bigserial not null constraint raw_json_data_pke primary key,
116 created_at timestamp with time zone,
117 deleted_at timestamp with time zone,
118 data jsonb,
119 url text
120);
121str := string(data)
122str = strings.ReplaceAll(str, `\u0000`, "")
123
Community Discussions contain sources that include Stack Exchange Network
QUESTION
How to insert JSON containing escape codes into a JSONB column in PostgreSQL using GORM
Asked 2021-Jul-08 at 18:16I'm trying to store the JSON bytes to PostgreSQL, but there's a problem.
\u0000 cannot be converted to text.
As you can see below, the JSON contains escape sequences such as \u0000
, which it seems PostgreSQL is interpreting as unicode characters, not JSON strings.
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12
Origin API data can be access form Here. There is \u0000 in it:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106
The struct for save is:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106type RawJSONData struct {
107 ID uint64 `gorm:"primarykey" json:"id"`
108 CreatedAt time.Time `json:"-"`
109 DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
110 Data datatypes.JSON `json:"data"`
111 URL string `gorm:"index" json:"url"`
112}
113
datatypes.JSON
is from gorm.io/datatypes
. It seems just is json.RawMessage
, it is (extend from?) a []byte
.
I use PostgreSQL's JSONB
type for storage this data.
Table:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106type RawJSONData struct {
107 ID uint64 `gorm:"primarykey" json:"id"`
108 CreatedAt time.Time `json:"-"`
109 DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
110 Data datatypes.JSON `json:"data"`
111 URL string `gorm:"index" json:"url"`
112}
113create table raw_json_data
114(
115 id bigserial not null constraint raw_json_data_pke primary key,
116 created_at timestamp with time zone,
117 deleted_at timestamp with time zone,
118 data jsonb,
119 url text
120);
121
ANSWER
Answered 2021-Jul-08 at 16:05Well, I solved it by:
1err := raws.SaveRawData(data, url)
2// if there is "\u0000" in the bytes
3if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
4 // try to remove \u0000, but not work
5 data = bytes.Trim(data, "\u0000")
6 e := raws.SaveRawData(data, url) // save data again
7 if e != nil {
8 return e // return the same error
9 }
10 return nil
11}
12{
13 "code": 0,
14 "message": "0",
15 "ttl": 1,
16 "data": {
17 "bvid": "BV1jb411C7m3",
18 "aid": 42443484,
19 "videos": 1,
20 "tid": 172,
21 "tname": "手机游戏",
22 "copyright": 1,
23 "pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
24 "title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
25 "pubdate": 1549100438,
26 "ctime": 1549100438,
27 "desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
28 "desc_v2": [
29 {
30 "raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
31 "type": 1,
32 "biz_id": 0
33 }
34 ],
35 "state": 0,
36 "duration": 265,
37 "rights": {
38 "bp": 0,
39 "elec": 0,
40 "download": 1,
41 "movie": 0,
42 "pay": 0,
43 "hd5": 0,
44 "no_reprint": 1,
45 "autoplay": 1,
46 "ugc_pay": 0,
47 "is_cooperation": 0,
48 "ugc_pay_preview": 0,
49 "no_background": 0,
50 "clean_mode": 0,
51 "is_stein_gate": 0
52 },
53 "owner": {
54 "mid": 39699039,
55 "name": "明眸-雅望",
56 "face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
57 },
58 "stat": {
59 "aid": 42443484,
60 "view": 1107,
61 "danmaku": 7,
62 "reply": 22,
63 "favorite": 5,
64 "coin": 4,
65 "share": 0,
66 "now_rank": 0,
67 "his_rank": 0,
68 "like": 10,
69 "dislike": 0,
70 "evaluation": "",
71 "argue_msg": ""
72 },
73 "dynamic": "#崩坏3#",
74 "cid": 74479750,
75 "dimension": {
76 "width": 1280,
77 "height": 720,
78 "rotate": 0
79 },
80 "no_cache": false,
81 "pages": [
82 {
83 "cid": 74479750,
84 "page": 1,
85 "from": "vupload",
86 "part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
87 "duration": 265,
88 "vid": "",
89 "weblink": "",
90 "dimension": {
91 "width": 1280,
92 "height": 720,
93 "rotate": 0
94 }
95 }
96 ],
97 "subtitle": {
98 "allow_submit": false,
99 "list": []
100 },
101 "user_garb": {
102 "url_image_ani_cut": ""
103 }
104 }
105}
106type RawJSONData struct {
107 ID uint64 `gorm:"primarykey" json:"id"`
108 CreatedAt time.Time `json:"-"`
109 DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
110 Data datatypes.JSON `json:"data"`
111 URL string `gorm:"index" json:"url"`
112}
113create table raw_json_data
114(
115 id bigserial not null constraint raw_json_data_pke primary key,
116 created_at timestamp with time zone,
117 deleted_at timestamp with time zone,
118 data jsonb,
119 url text
120);
121str := string(data)
122str = strings.ReplaceAll(str, `\u0000`, "")
123