-
Notifications
You must be signed in to change notification settings - Fork 10
feat: vendor Mustache templating engine & add Interpolator (AIC-2695) #172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
b94e0a1
0cc05bb
6406953
445633f
06e32ba
c768a52
b1b318e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| This product includes vendored third-party source code. The relevant licenses | ||
| and copyright notices are reproduced below. | ||
|
|
||
| ================================================================================ | ||
| JMustache (jmustache) | ||
| -------------------------------------------------------------------------------- | ||
| Vendored from: com.samskivert:jmustache:1.15 | ||
| Upstream: https://github.com/samskivert/jmustache | ||
| Location: src/main/java/com/launchdarkly/sdk/server/ai/internal/mustache/ | ||
|
|
||
| The JMustache source has been relocated into the internal, non-public package | ||
| com.launchdarkly.sdk.server.ai.internal.mustache and is compiled from source as | ||
| part of this library. Aside from the relocated package declaration and a short | ||
| provenance banner at the top of each file, the source is unmodified from the | ||
| upstream 1.15 release. | ||
|
|
||
| License: The (New) BSD License (BSD 3-Clause) | ||
|
|
||
| Copyright (c) 2010, Michael Bayne | ||
| All rights reserved. | ||
|
|
||
| Redistribution and use in source and binary forms, with or without | ||
| modification, are permitted provided that the following conditions are met: | ||
|
|
||
| * Redistributions of source code must retain the above copyright notice, | ||
| this list of conditions and the following disclaimer. | ||
| * Redistributions in binary form must reproduce the above copyright notice, | ||
| this list of conditions and the following disclaimer in the documentation | ||
| and/or other materials provided with the distribution. | ||
| * The name Michael Bayne may not be used to endorse or promote products | ||
| derived from this software without specific prior written permission. | ||
|
|
||
| THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND | ||
| ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
| WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
| DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE | ||
| FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL | ||
| DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR | ||
| SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | ||
| CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, | ||
| OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
| OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| ================================================================================ | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| package com.launchdarkly.sdk.server.ai.internal; | ||
|
|
||
| import com.launchdarkly.sdk.LDContext; | ||
| import com.launchdarkly.sdk.server.ai.internal.mustache.Mustache; | ||
| import com.launchdarkly.sdk.server.ai.internal.mustache.Template; | ||
|
|
||
| import java.util.HashMap; | ||
| import java.util.Map; | ||
| import java.util.concurrent.ConcurrentHashMap; | ||
|
|
||
| /** | ||
| * Renders AI Config message and instruction templates using Mustache, following the cross-SDK | ||
| * interpolation policy shared with other SDKs: | ||
| * <ul> | ||
| * <li><b>No HTML escaping.</b> The escape function is the identity, so {@code {{x}}} and | ||
| * {@code {{{x}}}} render identically and values are emitted verbatim.</li> | ||
| * <li><b>Missing and null variables render as the empty string</b> rather than throwing or | ||
| * leaving the placeholder in place.</li> | ||
| * <li><b>The reserved {@code ldctx} variable</b> is derived from the evaluation context and is | ||
| * merged in last, so it always overrides any caller-supplied {@code ldctx}. Context | ||
| * attributes are addressable as {@code {{ldctx.key}}}, {@code {{ldctx.name}}}, and so on.</li> | ||
| * </ul> | ||
| * <p> | ||
| * Compiled templates are cached, keyed by template text. The class is thread-safe: the Mustache | ||
| * compiler is immutable once configured, compiled {@link Template}s are safe for concurrent | ||
| * execution, and the cache is a {@link ConcurrentHashMap}. | ||
| * <p> | ||
| * This class is an internal implementation detail and is not part of the supported API. | ||
| */ | ||
| public final class Interpolator { | ||
| private final Mustache.Compiler compiler; | ||
| private final ConcurrentHashMap<String, Template> templateCache = new ConcurrentHashMap<>(); | ||
|
|
||
| /** | ||
| * Creates an interpolator with the cross-SDK escaping policy. | ||
| */ | ||
| public Interpolator() { | ||
| // defaultValue("") makes both missing variables and variables that resolve to null render as | ||
| // the empty string (it sets jmustache's missingIsNull=true and nullValue=""). escapeHTML(false) | ||
| // emits values verbatim, matching the JS/Python SDKs. | ||
| this.compiler = Mustache.compiler() | ||
| .escapeHTML(false) | ||
| .defaultValue(""); | ||
| } | ||
|
|
||
| /** | ||
| * Renders a template with the given variables and evaluation context. | ||
| * | ||
| * @param template the template text; if {@code null} the result is {@code null} | ||
| * @param variables caller-supplied variables; may be {@code null} | ||
| * @param context the evaluation context, exposed to the template as {@code ldctx}; may be | ||
| * {@code null} | ||
| * @return the rendered string, or {@code null} if {@code template} is {@code null} | ||
| */ | ||
| public String interpolate(String template, Map<String, Object> variables, LDContext context) { | ||
| if (template == null) { | ||
| return null; | ||
| } | ||
| Map<String, Object> merged = new HashMap<>(); | ||
| if (variables != null) { | ||
| merged.putAll(variables); | ||
| } | ||
| // ldctx is added last so it always wins over any caller-supplied "ldctx" entry. | ||
| merged.put("ldctx", contextToMap(context)); | ||
| return render(template, merged); | ||
| } | ||
|
|
||
| /** | ||
| * Renders a template with an already-assembled variable map (no {@code ldctx} injection). | ||
| * | ||
| * @param template the template text; if {@code null} the result is {@code null} | ||
| * @param variables the variables; may be {@code null} | ||
| * @return the rendered string, or {@code null} if {@code template} is {@code null} | ||
| */ | ||
| public String interpolate(String template, Map<String, Object> variables) { | ||
| if (template == null) { | ||
| return null; | ||
| } | ||
| return render(template, variables == null ? new HashMap<String, Object>() : variables); | ||
| } | ||
|
|
||
| private String render(String template, Map<String, Object> variables) { | ||
| Template compiled = templateCache.computeIfAbsent(template, compiler::compile); | ||
| return compiled.execute(variables); | ||
| } | ||
|
|
||
| /** | ||
| * Encodes the evaluation context directly into the nested map structure exposed to templates as | ||
| * {@code ldctx}, without round-tripping through JSON serialization. A single-kind context becomes | ||
| * a map of its attributes; a multi-kind context becomes | ||
| * {@code {"kind":"multi", "key":<fully-qualified key>, <kind>: {...}}} with one nested map per | ||
| * individual context. | ||
| */ | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Had AI double check this against other impls. Looks like .NET always emits anonymous true/false and not only when true. Also .NET puts the fully qualified key next to Multi at the top level.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also need to discuss with SDK team if private attributes should be going into the templating space. That will happen in a few hours.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @tanderson-ld what decision came out of the discussion with the SDK team
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good to template with those attributes. Did the .NET disparity get investigated / addressed?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, investigated it against the .NET source (ConfigFactory.cs) and addressed both items in this PR - good catch |
||
| private static Map<String, Object> contextToMap(LDContext context) { | ||
| if (context == null || !context.isValid()) { | ||
| return new HashMap<>(); | ||
| } | ||
| if (context.isMultiple()) { | ||
| Map<String, Object> map = new HashMap<>(); | ||
| map.put("kind", "multi"); | ||
| map.put("key", context.getFullyQualifiedKey()); | ||
| int count = context.getIndividualContextCount(); | ||
| for (int i = 0; i < count; i++) { | ||
| LDContext individual = context.getIndividualContext(i); | ||
| if (individual != null) { | ||
| // Mirror LaunchDarkly's standard context JSON: the per-kind objects nested under a | ||
| // multi-kind context omit "kind" because it is already implied by the property key. | ||
| map.put(individual.getKind().toString(), singleContextToMap(individual, false)); | ||
| } | ||
|
cursor[bot] marked this conversation as resolved.
|
||
| } | ||
| return map; | ||
| } | ||
| return singleContextToMap(context, true); | ||
| } | ||
|
|
||
| private static Map<String, Object> singleContextToMap(LDContext context, boolean includeKind) { | ||
| Map<String, Object> map = new HashMap<>(); | ||
| if (includeKind) { | ||
| map.put("kind", context.getKind().toString()); | ||
| } | ||
| map.put("key", context.getKey()); | ||
| if (context.getName() != null) { | ||
| map.put("name", context.getName()); | ||
| } | ||
| map.put("anonymous", context.isAnonymous()); | ||
| // Custom attribute values can be arbitrary JSON; convert each LDValue to a plain Java value | ||
| // (depth-capped) so nested objects/arrays remain addressable from templates. | ||
| for (String attribute : context.getCustomAttributeNames()) { | ||
| map.put(attribute, LDValueConverter.toJavaObject(context.getValue(attribute))); | ||
| } | ||
| return map; | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,180 @@ | ||
| // Vendored from com.samskivert:jmustache:1.15 (BSD 3-Clause, Copyright (c) 2010 Michael Bayne). | ||
| // Relocated to com.launchdarkly.sdk.server.ai.internal.mustache for supply-chain hardening (AIC-2695). | ||
| // Upstream: https://github.com/samskivert/jmustache -- unmodified except for this banner and the package | ||
| // declaration below. See THIRD-PARTY-NOTICES.txt for the full license text. | ||
| // | ||
| // | ||
| // JMustache - A Java implementation of the Mustache templating language | ||
| // http://github.com/samskivert/jmustache/blob/master/LICENSE | ||
|
|
||
| package com.launchdarkly.sdk.server.ai.internal.mustache; | ||
|
|
||
| import java.util.Collections; | ||
| import java.util.Iterator; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
| import java.util.NoSuchElementException; | ||
|
|
||
| /** | ||
| * A collector that does not use reflection and can be used with GWT. | ||
| */ | ||
| public abstract class BasicCollector implements Mustache.Collector | ||
| { | ||
| public Iterator<?> toIterator (final Object value) { | ||
| if (value instanceof Iterable<?>) { | ||
| return ((Iterable<?>)value).iterator(); | ||
| } | ||
| if (value instanceof Iterator<?>) { | ||
| return (Iterator<?>)value; | ||
| } | ||
| if (value.getClass().isArray()) { | ||
| final ArrayHelper helper = arrayHelper(value); | ||
| return new Iterator<Object>() { | ||
| private int _count = helper.length(value), _idx; | ||
| @Override public boolean hasNext () { return _idx < _count; } | ||
| @Override public Object next () { return helper.get(value, _idx++); } | ||
| @Override public void remove () { throw new UnsupportedOperationException(); } | ||
| }; | ||
| } | ||
| return null; | ||
| } | ||
|
|
||
| public Mustache.VariableFetcher createFetcher (Object ctx, String name) { | ||
| if (ctx instanceof Mustache.CustomContext) return CUSTOM_FETCHER; | ||
| if (ctx instanceof Map<?,?>) return MAP_FETCHER; | ||
|
|
||
| // if the name looks like a number, potentially use one of our 'indexing' fetchers | ||
| char c = name.charAt(0); | ||
| if (c >= '0' && c <= '9') { | ||
| if (ctx instanceof List<?>) return LIST_FETCHER; | ||
| if (ctx instanceof Iterator<?>) return ITER_FETCHER; | ||
| if (ctx.getClass().isArray()) return arrayHelper(ctx); | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| /** This should return a thread-safe map, either {@link Collections#synchronizedMap} called on | ||
| * a standard {@link Map} implementation or something like {@code ConcurrentHashMap}. */ | ||
| public abstract <K,V> Map<K,V> createFetcherCache (); | ||
|
|
||
| protected static ArrayHelper arrayHelper (Object ctx) { | ||
| if (ctx instanceof Object[]) return OBJECT_ARRAY_HELPER; | ||
| if (ctx instanceof boolean[]) return BOOLEAN_ARRAY_HELPER; | ||
| if (ctx instanceof byte[]) return BYTE_ARRAY_HELPER; | ||
| if (ctx instanceof char[]) return CHAR_ARRAY_HELPER; | ||
| if (ctx instanceof short[]) return SHORT_ARRAY_HELPER; | ||
| if (ctx instanceof int[]) return INT_ARRAY_HELPER; | ||
| if (ctx instanceof long[]) return LONG_ARRAY_HELPER; | ||
| if (ctx instanceof float[]) return FLOAT_ARRAY_HELPER; | ||
| if (ctx instanceof double[]) return DOUBLE_ARRAY_HELPER; | ||
| return null; | ||
| } | ||
|
|
||
| protected static final Mustache.VariableFetcher CUSTOM_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| Mustache.CustomContext custom = (Mustache.CustomContext)ctx; | ||
| Object val = custom.get(name); | ||
| return val == null ? Template.NO_FETCHER_FOUND : val; | ||
| } | ||
| @Override public String toString () { | ||
| return "CUSTOM_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static final Mustache.VariableFetcher MAP_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| Map<?,?> map = (Map<?,?>)ctx; | ||
| if (map.containsKey(name)) return map.get(name); | ||
| // special case to allow map entry set to be iterated over | ||
| if ("entrySet".equals(name)) return map.entrySet(); | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| @Override public String toString () { | ||
| return "MAP_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static final Mustache.VariableFetcher LIST_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| try { | ||
| return ((List<?>)ctx).get(Integer.parseInt(name)); | ||
| } catch (NumberFormatException nfe) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } catch (IndexOutOfBoundsException e) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| } | ||
| @Override public String toString () { | ||
| return "LIST_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static final Mustache.VariableFetcher ITER_FETCHER = new Mustache.VariableFetcher() { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| try { | ||
| Iterator<?> iter = (Iterator<?>)ctx; | ||
| for (int ii = 0, ll = Integer.parseInt(name); ii < ll; ii++) iter.next(); | ||
| return iter.next(); | ||
| } catch (NumberFormatException nfe) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } catch (NoSuchElementException e) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| } | ||
| @Override public String toString () { | ||
| return "ITER_FETCHER"; | ||
| } | ||
| }; | ||
|
|
||
| protected static abstract class ArrayHelper implements Mustache.VariableFetcher { | ||
| public Object get (Object ctx, String name) throws Exception { | ||
| try { | ||
| return get(ctx, Integer.parseInt(name)); | ||
| } catch (NumberFormatException nfe) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } catch (ArrayIndexOutOfBoundsException e) { | ||
| return Template.NO_FETCHER_FOUND; | ||
| } | ||
| } | ||
| public abstract int length (Object ctx); | ||
| protected abstract Object get (Object ctx, int index); | ||
| } | ||
|
|
||
| protected static final ArrayHelper OBJECT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((Object[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((Object[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper BOOLEAN_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((boolean[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((boolean[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper BYTE_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((byte[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((byte[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper CHAR_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((char[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((char[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper SHORT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((short[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((short[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper INT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((int[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((int[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper LONG_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((long[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((long[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper FLOAT_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((float[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((float[])ctx).length; } | ||
| }; | ||
| protected static final ArrayHelper DOUBLE_ARRAY_HELPER = new ArrayHelper() { | ||
| @Override protected Object get (Object ctx, int index) { return ((double[])ctx)[index]; } | ||
| @Override public int length (Object ctx) { return ((double[])ctx).length; } | ||
| }; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was only added to give attribution to the vendored lib but if theres existing attribution process, I'm happy to switch to that.