Yichun Zhang (agentzh)

Founder & CEO of OpenResty Inc., Creator of OpenResty, Meditator

[Yichun Zhang (agentzh) @agentzh]

And the project ban had never happened with the older Gemini 1.5 Flash model. Really need to fix your safety and security filters, otherwise, this thing can never be truly production ready for us. Alas.

[Yichun Zhang (agentzh) @agentzh]

@OfficialLoganK I really like #Gemini 2.0 Flash, but it keeps banning my GCP projects, saying my API uses violate Google's terms of service and other policies without providing any details. And I have to keep appealing every few days. It's very annoying and frustrating. All I've been doing is feeding it with open-source software source code and Gemini 2.0 Flash's own output... This never happened with the older Gemini 1.5 Flash model.
Furthermore, the API sometimes returns citation errors even if I disable all the safety filters. Fortunately, in that case, it at least quotes the offending text, which is often just the same model's own output. This is quite ironic.
Otherwise, Gemini 2.0 Flash is a great model. :) Hopefully, you guys can solve these issues soon. Thanks! #LLM

[Yichun Zhang (agentzh) @agentzh]

Today, I tried a more complex test case that required multi-step reasoning. The newly released #Gemini 2.0 Flash Thinking model (01-21 version) was disappointing and significantly inferior to the o1-mini. The critical issue was that it couldn't even generate XML/HTML tags correctly - <foo> became ```foo>. Alas. Meanwhile, Claude 3.5 sonnet, through prompt engineering, can achieve results close to o1-mini's performance. After the holiday, I plan to try Deepseek R1 with a few H100/H200 GPUs. #LLM

[Yichun Zhang (agentzh) @agentzh]

Recently, I accidentally noticed that Google Gemini's large language model API quietly changed its JSON format for returning citation meta error messages, so I had to modify my own API client code accordingly. Such backward-incompatible API changes are pretty annoying. Of course, Google's citation error checking for references and copyrighted text is exceptionally frustrating in itself. I haven't encountered such issues with other large language models. #Gemini #LLM #API

[Yichun Zhang (agentzh) @agentzh]

@gengjiawen I use them for almost everything: reading, learning, coding, testing, marketing, customer support, investor reports, legal contracts, everything :)

[Yichun Zhang (agentzh) @agentzh]

The cost-effectiveness of the Gemini 1.5 Flash model API is quite good, especially in batch processing mode (although its batch API is difficult to use). However, Google's cheapest Gemini 1.5 Flash 8b model is too weak—so weak that it sometimes doesn't follow instructions, causing many of our test cases to fail, and it's not easy to work around these issues.

In my daily AI application development, I really enjoy using the 1.5 Flash model for testing and debugging—it's fast and affordable.

The experimental Gemini 2.0 Flash model shows significantly better quality than 1.5 Flash. I hope the pricing won't increase after it becomes generally available (GA). #LLM

[Yichun Zhang (agentzh) @agentzh]

I recently noticed that the concurrency of large model batch processing tasks in each Google Cloud region has dropped from 4 to just 1. This is true for all Asia-Pacific and North American regions at least. The change happened just a few days ago. Perhaps they're preparing computing resources for the GA release of the Gemini 2.0 Flash new model? The official documentation states that the regional batch processing concurrency is 3.

I'm looking forward to Gemini 2.0 GA. Currently, the API rate limits for the experimental model are too strict, and the computing power is too low - the 1500 RPD (requests per day) isn't enough to run our test suite. Running the test suite once requires at least tens of thousands of LLM requests. #Gemini #LLM

[OpenResty @OpenResty]

Our OpenResty XRay dynamic tracing product's Java analyzers finally support Azul's Zulu JDK!

This includes Java-land CPU/off-CPU/diskIO flame graphs, latency analysis and smart packet capture, dynamic probes for Java method entry points, and more. We have a cryptocurrency exchange client who needs this explicitly. OpenJDK, Oracle JDK, and Zulu JDK—all three JVM variants are now fully supported. Depending on our enterprise customers' actual needs, we'll consider supporting other variants like IBM JDK and Microsoft's JDK later. #DynamicTracing #Java #JVM #ZuluJDK #Azul #eBPF

[OpenResty @OpenResty]

Our OpenResty XRay, a dynamic-tracing product, transforms black-box production systems into white-box ones. Many issues can be observed and captured before they escalate into harmful problems. This continuous insight and control over production systems is what's most crucial. Offline simulated traffic and testing methods often have limited coverage. No matter how much effort is invested in offline simulation, gaps will always exist.

Additionally, many of our customers prefer using OpenResty XRay for white-box analysis even in offline environments; regardless of the environment, white-box is much more convenient than black-box. You can't keep adding monitoring metrics and debug logs every time, can you? ;)

Our ultimate goal is to help customers quickly understand their legacy systems (or "sh*t mountain systems") and improve them efficiently at low cost rather than just being a tool for solving production issues. "Turning sh*t into gold" - that's what makes this truly interesting and valuable.

[Yichun Zhang (agentzh) @agentzh]

Many users ask me about the implementation principles and technology behind @OpenResty XRay, as it looks magical.

• For those interested, you can refer to my previous article about dynamic tracing: https://t.co/j8pQzctukK
• Here's another series of articles I wrote introducing the ylang language: https://t.co/ihDZno69nf
• And here's an article I wrote about the y++ language: https://t.co/X1NWPoruwh

Our recent advanced dynamic tracing capabilities for Java are implemented using our C++-compatible y++ language.

Interestingly, our team has been using OpenResty XRay to analyze OpenResty XRay itself. However, this capability is disabled for our customers to prevent misuse.
#DynamicTracing #DomainSpecificLanguages #eBPF

[OpenResty @OpenResty]

OpenResty XRay now supports setting dynamic probes at any #Java function entry point! This feature has wide applications - for example, we've implemented real-time latency statistics for requests within Tomcat servers based on this feature, and integrated it with our online intelligent packet capture analyzer. It doesn't depend on any configuration of the target process or log files.

It supports both interpreted and JIT-compiled Java methods. It's still based on 100% non-intrusive dynamic tracing technology, with negligible performance overhead.

It doesn't rely on bytecode injection or machine code injection, doesn't depend on JVM Agent mechanism, and is completely read-only for the JVM target process - the target process doesn't even need to restart. It supports both the latest Linux kernels and old Red Hat 7's 3.10 kernel.

Since this feature is very new, it currently only supports OpenJDK, but will soon support Oracle JDK. In the near future, it will definitely support more variants like Zulu JDK, IBM JDK, Azul JDK, and others.

Support for dynamic probes at Java function exit points will be added later.

#JVM #java #DynamicTracing #eBPF

[Yichun Zhang (agentzh) @agentzh]

Asahi Fedora Linux runs very well on my M2 Ultra Mac Studio. All devices including USB-C, HDMI, 10G network card, etc., work without any issues. It's very stable. In comparison, Linux support for Qualcomm Snapdragon X Elite devices is far behind. Thanks to the great work from the Asahi team. Previously, running Linux VMs through qemu HVF on macOS had too many problems and poor performance.
Going forward, our team will rely on Asahi Linux for development work targeting the Aarch64/ARM64 architecture.

[OpenResty @OpenResty]

Following OpenJDK, our OpenResty XRay can now deeply analyze #OracleJDK!
We're still using our 100% non-intrusive dynamic tracing technology. The analysis covers everything from the C++ VM level to the #Java source code level without any blind spots.

Next up will be other JDK variants like Zulu JDK and IBM JDK 😀
#JVM #java #DynamicTracing #eBPF

[Yichun Zhang (agentzh) @agentzh]

Some people speculate that our OpenResty XRay uses the ELF file's build ID to look up existing DWARF information in a database, which made me laugh so hard.

We're actually targeting binaries that customers have modified and compiled without the -g flag or binary software packages downloaded from third parties without debug symbols. This is the most challenging and general problem to solve.

For any existing binary programs and their debug symbols found online, our crawler system has already indexed them in our Pkg DB, which has accumulated hundreds of terabytes of data and is still growing daily. Why would we need such advanced machine learning technology for those cases?

[OpenResty @OpenResty]

🔥 OpenResty XRay's New Capability Unlocked! Analyze Applications Without Debug Symbols! Video: https://t.co/LwVOEX5vsR

Even when binary programs have no debug information, it can:
• Automatically reconstruct debug symbols from binary programs using our own machine-learning algorithms
• Display complete call stack traces at the language levels of C, Lua, Python, Perl, PHP, etc.
• Precisely locate source files and line numbers
• Zero configuration and zero hints needed for target programs. 100% non-intrusive.

The accuracy here is close to 100%, at least for the types of symbols our product needs; they are primarily accurate enough. If there are any inaccurate cases, our human experts can help fine-tune them, which is not a problem.

We are porting this magical new technology to large C++ program binaries like the JVM. OpenResty XRay will soon be able to automatically reconstruct high-quality debug symbols from any JVM binary program files. It should take about a few more days.

Text version: https://t.co/oBlfz1aMZa

#eBPF #OpenResty #DynamicTracing #MachineLearning #DebugSymbols

[Yichun Zhang (agentzh) @agentzh]

Someone on the Internet asked: "What if you randomly grab a programmer from the street, and they can solve problems using your OpenResty XRay tools?" They seem to be concerned that only experts could use our products. My response is that their worry is unnecessary, and our technical experts will directly help clients solve problems - that's our guarantee. It doesn't matter if clients themselves don't understand technology or know how to use OpenResty XRay preciselyproducts. This is exactly why some non-technical business owners like our services. As the saying goes, "You make a wish, we make it happen!"

[Yichun Zhang (agentzh) @agentzh]

The open-source vector database Qdrant actually has a nasty bug that occasionally causes unexpected data loss. The official team finally confirmed it was a bug in the database index implementation. Alas. https://t.co/uwHkuzIFt2 There are so many bugs in the software world... a database that loses data is the scariest thing.

Fortunately, it got fixed in the latest version.

[Yichun Zhang (agentzh) @agentzh]

One of my dreams for many years has been to turn all tech stacks and applications in all programming languages from black boxes into white boxes, eliminating mysterious issues in production environments. This is precisely the mission of our OpenResty XRay.

Previously, we encountered a case where a hardware bug was discovered using OpenResty XRay on a customer's production Xeon CPU. Intel engineers initially didn't acknowledge it was a hardware issue and insisted on patching the Linux kernel to confirm the problem. Dramatically, OpenResty XRay then discovered bugs in their kernel patch. Finally, it was confirmed to be a CPU hardware issue. We still need to thank the Intel engineers for helping confirm the problem.

There are so many bugs in the world, so OpenResty XRay aims to be the best bug hunter.

Going further, hopefully, one day, it can even automatically apply binary patches to VMs or machine code in running processes to fix bugs or security vulnerabilities. This belongs to live patching. Everything can be live-patched from kernel to system software and up to business code without even restarting processes. We have already made some exciting experiments in this direction, and it's very promising. Advanced technology looks like magic 😁