whoisltd Personal Knowledge Base

vLLM is an open-source inference engine for Large Language Models (LLMs) designed for high-throughput production environments. Key technical features include:

• Continuous Batching: Processes multiple requests in a single GPU forward pass to increase throughput.
• PagedAttention: Optimizes KV cache memory management to reduce fragmentation.
• OpenAI-compatible API: Provides a drop-in replacement endpoint for easy integration.
• Quantization Support: Supports INT8 and INT4 to reduce VRAM requirements.

Natural Language Processing (NLP) involves several core tasks, often starting with Language Modeling as a foundation. Key tasks include:

• Text Classification
• Information Extraction
• Information Retrieval
• Question Answering
• Machine Translation
• Conversational Agents

NLP is considered challenging due to human language ambiguity, heavy dependence on context, idioms, and multilingualism.

Use different phrasing depending on whether you are asking about a person or an object. For buildings, you can specifically ask about the number of stories.

For Objects:

• Singular: "How tall is it?", "What's the height?", "How many stories is it?" (buildings)
• Plural: "How tall are they?", "How high are they?"

For People:

• Singular: "How tall are you?", "What's your height?"
• Plural: "How tall are they?"

Common Response Patterns:

• Full detail: "I'm 5 feet, 4 inches tall"
• Shorthand: "I'm 5'11""
• Casual/Elliptical: "He's six one" (omitting 'feet' and 'inches')

When asking about weight, use the following structures. Note that asking about a person's weight is generally considered more sensitive than asking about height.

For Objects:

• Singular: "How much does this weigh?", "What does this weigh?", "How heavy is this?"
• Plural: "How much do these weigh?", "What do these weigh?", "How heavy are these?"

For People:

• Singular: "How much do you weigh?", "How much does he/she weigh?", "What's your weight?"
• Plural: "How much do they weigh?", "What's their weight?"

Usage Constraints:

• Avoid saying "How much pounds?" or "What weight is it?"

Common Response Patterns:

• "I weigh 1912 pounds"
• "He weighs about 175 pounds"
• "It's about 10 pounds"

Use the following patterns to describe specific dates, days, and relative timeframes within a month:

Talking about specific days:

• Current day: "Today is the 18th", "Today's Thursday", "It's Thursday the 18th", or "It's April 18th".
• Relative days: "Yesterday was the 17th", "Tomorrow is the 19th", "Tomorrow is Friday", or "The day after tomorrow is Saturday".
• Month boundaries: "There are 20 days in the month of April", "April began on a Monday", or "April ends on a Tuesday".

Talking about weeks:

• Identifying a week: "This is the week of the 15th" (indicating the week starts on April 15).
• Scheduling: "I'll be in Chicago on the week of the 22nd. That's next week".
• Relative time: "Last week was not as busy" (referring to the previous week), "exactly one week ago on April [date]", "A week from now will be April 25", or "It will be May in about two weeks".