This introduces some very clever optimizations to speedup semgrep by
more than 2! The algorithms are quite complicated, look at the code.

test plan:
OMG, we shipped a bytecode program all this time ...

BEFORE:

semgrep-core -test_rules ~/semgrep-rules: 20s
make test in semgrep-rules: 2m51s (real)

perf/run-benchmarks --small-only --std-only: 2m33 (real)
semgrep.bench.zulip.std.duration = 7.039 s
semgrep.bench.dropbox.std.duration = 3.460 s
semgrep.bench.coinbase.std.duration = 17.352 s
semgrep.bench.apache.std.duration = 12.362 s
semgrep.bench.0c34.std.duration = 2.719 s
semgrep.bench.lodash.std.duration = 5.979 s
semgrep.bench.njs-old-dvna.std.duration = 19.079 s
semgrep.bench.DVWA.std.duration = 13.937 s
semgrep.bench.juice-shop.std.duration = 25.761 s
semgrep.bench.Vulnerable-Flask-App.std.duration = 14.952 s
semgrep.bench.pallets.std.duration = 10.450 s
semgrep.bench.socketio.std.duration = 5.246 s
semgrep.bench.coolMenu.std.duration = 4.512 s
semgrep.bench.t00sh.std.duration = 4.293 s
semgrep.bench.grpc.std.duration = 5.562 s

Main.exe size: 47MB
startup time (semgrep-core --help): 0.160s

AFTER:

semgrep-core -test_rules ~/semgrep-rules: 10s
make test in semgrep-rules:  2m36s (real)

perf/run-benchmarks --small-only --std-only: 1m15s
semgrep.bench.zulip.std.duration = 3.491 s
semgrep.bench.dropbox.std.duration = 1.517 s
semgrep.bench.coinbase.std.duration = 9.234 s
semgrep.bench.apache.std.duration = 5.977 s
semgrep.bench.0c34.std.duration = 1.422 s
semgrep.bench.lodash.std.duration = 3.271 s
semgrep.bench.njs-old-dvna.std.duration = 8.239 s
semgrep.bench.DVWA.std.duration = 6.460 s
semgrep.bench.juice-shop.std.duration = 13.027 s
semgrep.bench.Vulnerable-Flask-App.std.duration = 7.575 s
semgrep.bench.pallets.std.duration = 4.601 s
semgrep.bench.socketio.std.duration = 2.754 s
semgrep.bench.coolMenu.std.duration = 1.966 s
semgrep.bench.t00sh.std.duration = 2.947 s
semgrep.bench.grpc.std.duration = 2.820 s

Main.exe size: 94MB
startup time (semgrep-core --help): 0.068s

* Update to OCaml 4.12.0

This is required to compile semgrep on recent macOS.
This closes https://github.com/returntocorp/semgrep/issues/3158
This closes https://github.com/returntocorp/semgrep/issues/3027

test plan:
wait for CI failures if any

* update build instructions

* Pipe semgrep-core stderr to stderr when --debug

* fixup! Pipe semgrep-core stderr to stderr when --debug

* Update timing to 0.58 and docker image to more recent ocaml-layer

test plan:
wait for CI results

* switch to 4.10.2 also for osx-release.sh

* update to ocamlformat 0.16.0

This is needed if we want to use ppxlib 0.20 (itself needed
if we want to use OCaml 4.12.0, itself needed if we don't
want errors on macos HomeBrew)

test plan:
pre-commit run --all lint-ocaml

* fix CI

* comment and use latest ocaml-layer image which hopefully has

ocamlformat 0.16

* test

* more test

* TEMPORARY change of pull_request_target to pull_request to debug lint.yml

* misc

* update to ppxlib 0.20.0

This is needed to be able to compile pfff (and so semgrep) with
OCaml 4.12.0 (which itself is needed to compile correctly on Homebrew
and recent macOS)

This should help #3027

test plan:
make test

* update docker returntocorp/ocaml:alpine to one using 4.10.2

Mostly just put underscore in front of unused variable

test plan:
make test

* refactor: combined fix/message string interpolation

* fix: fixed issue where similar metavar names could collide during interpolation

E.g., foo(bar).baz()

test plan:
make test # test included

* synthesizing: Pattern_from_Targets ExprStmt lists

Add support for targets that are ExprStmt lists in Pattern_from_Targets.

For example, two separate targets might be

```
x = req.query.foo;
exec(x);
```

and

```
y = req.query.foo;
exec(y);
```

yielding the pattern

```
$X = req.query.foo;
exec($X);
```

Limitations:
- Targets must be of equal length. Cannot yet generate ellipsis for
statements to equalize length of targets.
- No additional support for statements other than ExprStmt.

* Fix trailing whitespace.

* Remove use of ($)

1. Allow disabling the implicit ... for record patterns via `options:`.
2. Match "spread fields" (JS `...x`), or any other kind of field, at any
   position; same as we already do for definitions.

Closes #3120

test plan:
make test # tests included

* Translating tree-sitter-bash CST to a bash AST (WIP)

* More progress toward bash support

* Add support for bash simple commands

* Fix dune file after bad merge

* Fix bash parsing bug

* [Java] add generic parameters in AST

This closes https://github.com/returntocorp/semgrep/issues/3482

test plan:
test files included
$ yy -lang java -f tests/java/misc_generic.sgrep tests/java/misc_generic.java -pvar '$TYPE'
...
tests/java/misc_gentests/java/misc_generic.java:4: A<int>

* changelog update

* Adding bracket to type arguments in AST generic

This will help https://github.com/returntocorp/semgrep/issues/3482

test plan:
make test

* make regenerate-tests

* Experimental support for HTML

This does not rely on the generic (spacegrep) mode but
really parse HTML. This allows more semantic matching
(e.g., matching attributes in any order)

test plan:
test files included
make test

```
semgrep) [pad@yrax yy (semantic_html)]$ semgrep -l html -e '<html ...>...</html>' tests/html/
tests/html/misc_many_features.html
2:<html attr1="a" attr2="b" attr3="c" attr4="d" >some text</html>
--------------------------------------------------------------------------------
4:<html attr2="b" attr1="a" attr3="c" attr4="d" >some other text</html>
ran 1 rules on 2 files: 2 findings
```

* Example from https://github.com/returntocorp/semgrep/issues/1945

This kinda close #1945

* fix make test

Allow filtering based on a single `pattern` or `pattern-regex`, without
need for a parent `patterns` or `pattern-either`.

test plan:
semgrep --config semgrep-core/tests/OTHER/rules/metavar_pattern_lang2.yaml semgrep-core/tests/OTHER/rules/metavar_pattern_lang2.generic
  #^ now works

* Basic support for Vue files

This is the final PR in a series of PRs to add support for Vue.
For now, you can write Javascript patterns to match code
in the <script> part of Vue files. You can also
use JSX patterns to match code in the <template> part.

This closes #1751

Remaining todos:
 - parse JS expressions in templates
 - allow Vue template syntax, not JSX, to express HTML snippets
 - better error management when some JS code can not be parsed
 - more tests

test plan:
make test and
```
semgrep) [pad@yrax semgrep (vue_p3)]$ semgrep -l vue -e 'foo(...)' ~/yy/tests/vue/
/home/pad/yy/tests/vue/concrete_syntax.vue
4:    foo(1, 2);
--------------------------------------------------------------------------------
6:    foo(1,2);
--------------------------------------------------------------------------------
8:    foo (1, 2);
--------------------------------------------------------------------------------
10: foo(1,
11:     2);
--------------------------------------------------------------------------------
13: foo(1, // comment
14:     2);
--------------------------------------------------------------------------------
16: foo(2,1)

/home/pad/yy/tests/vue/parsing/single_line_script.vue
2:foo(1, 2)
ran 1 rules on 3 files: 7 findings
```

* regenerate tests

We are no longer restricted to single instruction/expression patterns.

Goodbye fake taint?

Helps #3199

test plan:
make test # tests included

This is a temporary fix.

test plan:
```
 /home/pad/yy/_build/default/src/cli/Main.exe -keep_tmp_files -lang vue -dump_ast tests/vue/parsing/basic.vue
[0.105  Info       Main.Dune__exe__Main ] loaded log_config.json
[0.105  Info       Main.Dune__exe__Main ] Executed as: /home/pad/yy/_build/default/src/cli/Main.exe -keep_tmp_files -lang vue -dump_ast tests/vue/parsing/basic.vue
[0.105  Info       Main.Dune__exe__Main ] Version: semgrep-core version: v0.57.0-30-g6f427678-dirty, pfff: 0.42
[0.105  Info       Main.Parse_target    ] trying to parse with TreeSitter parser tests/vue/parsing/basic.vue
[0.107  Info       Main.Parse_target    ] trying to parse with TreeSitter parser /tmp/tmp-30179-e34cc1.js
Pr(
  [ExprStmt(
     Xml(
       {xml_tag=XmlClassic((), ("template", ()), (), ()); xml_attrs=[
        ];
        xml_body=[XmlText(("  ", ()));
                  XmlXml(
                    {xml_tag=XmlClassic((), ("p", ()), (), ()); xml_attrs=[
                     ];
                     xml_body=[XmlText(("    Hello, ", ()));
                               XmlXml(
                                 {xml_tag=XmlClassic((), ("a", ()), (), ());
                                  xml_attrs=[XmlAttr((":", ()), (),
                                               L(String(("url", ()))))];
                                  xml_body=[XmlExpr(
                                              Some(L(String((" name ", ())))))];
                                  }); XmlText(("!  ", ()))];
                     }); XmlText(("", ()))];
        }), ());
   ExprStmt(
     Assign(
       DotAccess(OtherExpr(OE_Module, [Tk(())]), (),
         EN(
           Id(("exports", ()),
             {id_resolved=Ref(None); id_type=Ref(None);
              id_constness=Ref(None); }))), (),

```

* Parsing embedded JS in <script> in vue files

test plan:
```
pad@yrax yy (vue_p2)]$ yy -lang vue -dump_ast tests/vue/parsing/single_line_script.vue
+ /home/pad/yy/_build/default/src/cli/Main.exe -lang vue -dump_ast tests/vue/parsing/single_line_script.vue
[0.111  Info       Main.Dune__exe__Main ] loaded log_config.json
[0.111  Info       Main.Dune__exe__Main ] Executed as: /home/pad/yy/_build/default/src/cli/Main.exe -lang vue -dump_ast tests/vue/parsing/single_line_script.vue
[0.111  Info       Main.Dune__exe__Main ] Version: semgrep-core version: v0.57.0-26-g6a669dcb-dirty, pfff: 0.42
[0.112  Info       Main.Parse_target    ] trying to parse with TreeSitter parser tests/vue/parsing/single_line_script.vue
[0.112  Info       Main.Parse_target    ] trying to parse with TreeSitter parser /tmp/tmp-21439-4f2ff5.js
Pr(
  [ExprStmt(
     Call(
       N(
         Id(("foo", ()),
           {id_resolved=Ref(None); id_type=Ref(None); id_constne...

This reverts commit b0cf583c.

1. Generalize Rule type to absorb Tainting_rule.
2. Use same parser for search and taint rules.
3. Move tainting code into the engine library.
4. Let -config run taint rules, remove -tainting_rules_file
   and -dump_tainting_rules.
5. Print taint-mode matches to stdout if -json is not given.

Also, update Semgrep CLI accordingly.

Closes #3295

test plan:
make -C semgrep-core test
make -C semgrep test

This does not convert the <script>...</script> part
yet to JS. I'll do that in a separate PR

test plan:
```
+ /home/pad/yy/_build/default/src/cli/Main.exe -lang vue -dump_ast tests/vue/parsing/basic.vue
[0.107  Info       Main.Dune__exe__Main ] loaded log_config.json
[0.107  Info       Main.Dune__exe__Main ] Executed as: /home/pad/yy/_build/default/src/cli/Main.exe -lang vue -dump_ast tests/vue/parsing/basic.vue
[0.107  Info       Main.Dune__exe__Main ] Version: semgrep-core version: v0.57.0-25-gd5348a91-dirty, pfff: 0.42
[0.108  Info       Main.Parse_target    ] trying to parse with TreeSitter parser tests/vue/parsing/basic.vue
Pr(
  [ExprStmt(
     Xml(
       {xml_tag=XmlFragment((), ()); xml_attrs=[];
        xml_body=[XmlXml(
                    {xml_tag=XmlClassic((), ("template", ()), (), ());
                     xml_attrs=[];
                     xml_body=[XmlText(("  ", ()));
                               XmlXml(
                                 {xml_tag=XmlClassic((), ("p", ()), (), ());
                                  xml_attrs=[];
                                  xml_body=[XmlText(("    Hello, ", ()));
                                            XmlXml(
                                              {
                                               xml_tag=XmlClassic((),
                                                         ("a", ()), (), ());
                                               xml_attrs=[XmlAttr((":", ()),
                                                            (),
...
```

Closes https://github.com/returntocorp/semgrep/issues/3476

Turn off the `pattern: $X` optimization for now. To perform it properly, we need to get the sub-AST of the range produced by the other patterns in the `and` and run the pattern on that range.

Test plan: make test

* adding boilerplate for parsing Vue files

test plan:
make test

* misc

* fix semgrep-vue, do not ovverride html scanner

We just reuse the constructs we used for JSX/TSX/XHP.

test plan:
```
[pad@yrax yy (html2)]$ yy -lang html -dump_ast tests/html/parsing/hello.html
+ /home/pad/yy/_build/default/src/cli/Main.exe -lang html -dump_ast tests/html/parsing/hello.html
[0.104  Info       Main.Dune__exe__Main ] loaded log_config.json
[0.104  Info       Main.Dune__exe__Main ] Executed as: /home/pad/yy/_build/default/src/cli/Main.exe -lang html -dump_ast tests/html/parsing/hello.html
[0.105  Info       Main.Dune__exe__Main ] Version: semgrep-core version: v0.57.0-21-gad17ca23-dirty, pfff: 0.42
[0.105  Info       Main.Parse_target    ] trying to parse with TreeSitter parser tests/html/parsing/hello.html
Pr(
  [ExprStmt(
     Xml(
       {xml_tag=XmlFragment((), ()); xml_attrs=[];
        xml_body=[XmlXml(
                    {xml_tag=XmlClassic((), ("html", ()), (), ());
                     xml_attrs=[];
                     xml_body=[XmlText(("  ", ()));
                               XmlXml(
                                 {
                                  xml_tag=XmlClassic((), ("body", ()), (), ());
                                  xml_attrs=[];
                                  xml_body=[XmlText(("    Hello World ", ()));
                                            XmlXml(
                                              {
                                               xml_tag=XmlClassic((),
...
```