Skip to content

branch-3.1: [fix](es) Handle object type in Elasticsearch mapping when table is empty #56152 #56326

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 28, 2025
Merged

branch-3.1: [fix](es) Handle object type in Elasticsearch mapping when table is empty #56152 #56326

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docker/thirdparties/docker-compose/elasticsearch/scripts/es_init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,8 @@ curl "http://${ES_7_HOST}:9200/test2_20220808" -H "Content-Type:application/json
curl "http://${ES_7_HOST}:9200/test2_20220809" -H "Content-Type:application/json" -X PUT -d '@/mnt/scripts/index/es7_test2.json'
# create index test3_20231005
curl "http://${ES_7_HOST}:9200/test3_20231005" -H "Content-Type:application/json" -X PUT -d '@/mnt/scripts/index/es7_test3.json'
# create index test_object for object type testing (empty table)
curl "http://${ES_7_HOST}:9200/test_object" -H "Content-Type:application/json" -X PUT -d '@/mnt/scripts/index/es7_test_object.json'
# put data for tese1
curl "http://${ES_7_HOST}:9200/test1/_doc/1" -H "Content-Type:application/json" -X POST -d '@/mnt/scripts/data/data1.json'
curl "http://${ES_7_HOST}:9200/test1/_doc/2" -H "Content-Type:application/json" -X POST -d '@/mnt/scripts/data/data2.json'
Expand Down
16 changes: 16 additions & 0 deletions docker/thirdparties/docker-compose/elasticsearch/scripts/index/es7_test_object.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"properties": {
"id": {
"type": "integer"
},
"user_info": {
"type": "object"
}
}
}
}
2 changes: 2 additions & 0 deletions fe/fe-core/src/main/java/org/apache/doris/datasource/es/EsUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,8 @@ private static Column parseEsField(String fieldName, ObjectNode fieldValue, List
case "constant_keyword":
type = ScalarType.createStringType();
break;
// When ES table is empty, object fields still have explicit "type": "object" in mapping
case "object":
case "nested":
type = Type.JSONB;
break;
Expand Down
12 changes: 12 additions & 0 deletions regression-test/data/external_table_p0/es/test_es_query.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -611,6 +611,12 @@ string_ignore_above_10 text_ignore_above_10
4444
4444

-- !sql_7_39 --
id int Yes true \N
user_info json Yes true \N

-- !sql_7_40 --

-- !sql_7_50 --
value1 value2

Expand Down Expand Up @@ -1421,6 +1427,12 @@ string_ignore_above_10 text_ignore_above_10
4444
4444

-- !sql_7_39 --
id int Yes true \N
user_info json Yes true \N

-- !sql_7_40 --

-- !sql_7_50 --
value1 value2

Expand Down
2 changes: 2 additions & 0 deletions regression-test/suites/external_table_p0/es/test_es_query.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,8 @@ suite("test_es_query", "p0,external,es,external_docker,external_docker_es") {
order_qt_sql_7_36 """select test1, test2 from test1 where test2 like 'text%';"""
order_qt_sql_7_37 """select test9 from test1;"""
order_qt_sql_7_38 """select test9 from test2;"""
order_qt_sql_7_39 """desc test_object;"""
order_qt_sql_7_40 """select * from test_object;"""

List<List<String>> tables7N = sql """show tables"""
boolean notContainHide7 = true
Expand Down