Liang Hexiang's Space

website·Indexed 30 days ago

https://lianghexiang.github.io/space/

A personal knowledge base and technical blog covering a wide range of topics including PySpark tutorials, Large Language Models (LLM), Retrieval-Augmented Generation (RAG), prompt engineering, vector databases, and software development practices such as design patterns, gRPC, Python, MySQL, and Linux configuration.

Tokens: 10.3K
Snippets: 41
Records: 91
Agent score: 74%

What's inside Liang Hexiang's Space

Overview of PySpark for Distributed Data Processing
PySpark is the Python API for Apache Spark, designed to run Python applications across distributed clusters (multiple nodes) in parallel. It serves as an analytical processing engine for large-scale distributed data processing and machine learning applications. This specific guidance is based on Spark version 3.5.
https://lianghexiang.github.io/space/category/%E6%8A%80%E6%9C%AF/
Introduction to PySpark
PySpark is the Python API for Apache Spark, designed to run Python applications using Spark's distributed computing capabilities. It serves as an analytical processing engine for large-scale distributed data processing and machine learning applications, allowing programs to run in parallel across a distributed cluster of multiple nodes.
https://lianghexiang.github.io/space/
Introduction to PySpark
PySpark is the Python API for Apache Spark, designed for large-scale distributed data processing and machine learning applications. It allows Python applications to run in parallel across a distributed cluster of multiple nodes, leveraging the analytical processing engine of Apache Spark.
https://lianghexiang.github.io/space/tag/pyspark/
Introduction to PySpark for Distributed Data Processing
PySpark is the Python API for Apache Spark, designed to run Python applications on distributed clusters (multiple nodes) in parallel. It serves as an analytical processing engine for large-scale distributed data processing and machine learning applications. This specific guidance is based on Spark version 3.5.
https://lianghexiang.github.io/space/category/it/
Overview of Qwen2 (Tongyi Qianwen) open-source model
Qwen2, developed by Alibaba, is a decoder-only open-source model. It utilizes SwiGLU activation, RoPE (Rotary Position Embedding), and GQA (Grouped-Query Attention) architectures. It is noted for strong performance in Chinese language tasks.
https://lianghexiang.github.io/space/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/LLM/%E5%A4%A7%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%AE%80%E4%BB%8B.html
Overview of the Singleton Design Pattern
The Singleton pattern is a creational design pattern that ensures a class has only one instance while providing a global access point to that instance.
https://lianghexiang.github.io/space/tag/%E8%AE%BE%E8%AE%A1%E6%A8%A1%E5%BC%8F/
Overview of the Singleton Design Pattern
The Singleton pattern is a creational design pattern that ensures a class has only one instance while providing a global access point to that instance.
https://lianghexiang.github.io/space/category/python/
Introduction to PySpark for distributed data processing
PySpark is the Python API for Apache Spark, designed to run Python applications on distributed clusters (multiple nodes). It serves as an analytical processing engine for large-scale distributed data processing and machine learning applications. This specific guidance is based on Spark version 3.5.
https://lianghexiang.github.io/space/article/
Introduction to PySpark
PySpark is a Python API for Apache Spark, designed for large-scale distributed data processing and machine learning applications. It allows developers to run Python applications in parallel across a distributed cluster of multiple nodes, leveraging the Spark analysis processing engine.
https://lianghexiang.github.io/space/tag/%E5%A4%A7%E6%95%B0%E6%8D%AE/
Overview of other closed-source LLMs (Gemini, Wenxin Yiyan, Xinghuo)
Several other major closed-source models include:
- Google Gemini: Successor to PaLM 2, available in Ultra, Pro, and Nano versions.
- Baidu Wenxin Yiyan (ERNIE Bot): Knowledge-enhanced model. The free version uses Wenxin 3.5, while the professional version (paid) uses Wenxin 4.0.
- iFLYTEK Xinghuo: A cognitive large model supporting various NLP tasks, with version 3.5 supporting system instructions and plugin calls.
https://lianghexiang.github.io/space/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/LLM/%E5%A4%A7%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%AE%80%E4%BB%8B.html
Introduction to PySpark
PySpark is a Python library for Apache Spark that allows developers to run Python applications using Spark's capabilities. It serves as a Python API for a powerful distributed data processing and machine learning analysis engine, enabling applications to run in parallel across a distributed cluster of multiple nodes. This specific guidance is based on Spark version 3.5.
https://lianghexiang.github.io/space/tag/%E6%95%99%E7%A8%8B/
Overview of PySpark and Apache Spark
PySpark is the Python API for Apache Spark, an open-source unified analytics engine designed for large-scale data processing. It allows developers to run Python applications on distributed clusters (multiple nodes) to perform powerful data processing and machine learning tasks. Spark is designed to be faster than traditional MapReduce by utilizing in-memory computing and caching to reduce latency, especially for iterative machine learning algorithms and interactive analysis.
https://lianghexiang.github.io/space/posts/pyspark/PySpark%E5%88%9D%E5%AD%A6%E8%80%85%E6%95%99%E7%A8%8B.html