[CORE-4279] Wire up schema_registry/json

[andijcr]

Wire up schema_registry support for json schema.

this pr enables json schema inputs and adds a ducktape test for json schema compatibility

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v24.1.x
• v23.3.x
• v23.2.x

Release Notes

• none

[BenPope]

Please put a compatibility test (such as from the demo) into ducktape, along with something like this:

index fbb6b972b2..cc5de47f7d 100644
--- a/src/v/pandaproxy/schema_registry/sharded_store.cc
+++ b/src/v/pandaproxy/schema_registry/sharded_store.cc
@@ -686,12 +686,18 @@ ss::future<bool> sharded_store::is_compatible(
         co_return true;
     }
 
-    // Currently support PROTOBUF, AVRO
-    if (
-      new_schema.type() != schema_type::avro
-      && new_schema.type() != schema_type::protobuf) {
+    // Currently support JSON, PROTOBUF, AVRO
+    if (![type = new_schema.type()] {
+            switch (type) {
+            case schema_type::avro:
+            case schema_type::protobuf:
+            case schema_type::json:
+                return true;
+            }
+            return false;
+        }()) {
         throw as_exception(invalid_schema_type(new_schema.type()));
-    }
+    };
 
     // if transitive, search all, otherwise seach forwards from version
     if (

[BenPope]

🔥