Skip to content

feat: add tos handler#94

Merged
yaozheng-fang merged 13 commits intovolcengine:mainfrom
ZQlQZ:feature/toshandler
Aug 29, 2025
Merged

feat: add tos handler#94
yaozheng-fang merged 13 commits intovolcengine:mainfrom
ZQlQZ:feature/toshandler

Conversation

@ZQlQZ
Copy link
Copy Markdown
Contributor

@ZQlQZ ZQlQZ commented Aug 22, 2025

Add toshandler

@zakahan zakahan changed the title feat: add toshandler feat: add tos handler Aug 22, 2025
Comment thread veadk/database/tos/toshandler.py Outdated
Comment thread veadk/database/tos/toshandler.py Outdated
@zakahan zakahan self-requested a review August 22, 2025 09:20
Comment thread veadk/runner.py Outdated
url = messages.media
if self.agent.tracers:
tos_handler = TOSHandler()
url = tos_handler.gen_url(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build_url

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or build_index

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build_key

Comment thread veadk/runner.py Outdated
self.user_id, self.app_name, session_id, messages.media
)
asyncio.create_task(tos_handler.upload_to_tos(url, data, "bytes"))
tos_handler.close_client()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be need a wrapper for upload_to_tos to avoid manually close client?

Comment thread veadk/database/tos/tos_handler.py Outdated
class TOSHandler:
def __init__(self):
"""Initialize TOS configuration information"""
self.region = getenv("DATABASE_TOS_REGION")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

class TOSHandlerConfig(BaseModel)

Comment thread veadk/database/tos/tos_handler.py Outdated
return f".{candidate.lower()}"
return f".{parts[-1].lower()}"

def gen_url(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use build_url here, as other places use build

Comment thread veadk/database/tos/tos_client.py Outdated
config: TOSConfig = Field(default_factory=TOSConfig)
client: Optional[Any] = Field(default=None, description="TOS client instance")

def __init__(self, **data: Any):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

model_post_init

self._client

Comment thread veadk/database/tos/tos_client.py Outdated
self,
object_key: str,
data: Union[str, bytes],
data_type: Literal["file", "bytes"],
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this arg, check it by code

Comment thread veadk/database/tos/tos_client.py Outdated

def __init__(self, **data: Any):
super().__init__(**data)
self.client = self._init()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

health check

logger.info

Comment thread veadk/database/tos/tos_client.py Outdated
return False

def _do_upload_file(self, object_key: str, file_path: str) -> bool:
client = self._init_tos_client()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this

Comment thread veadk/runner.py Outdated
)

def _convert_messages(self, messages) -> list:
def _build_object_key(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_build_tos_object_key

@yaozheng-fang yaozheng-fang marked this pull request as draft August 26, 2025 08:01
@yaozheng-fang yaozheng-fang marked this pull request as ready for review August 29, 2025 09:53
@yaozheng-fang yaozheng-fang merged commit b9cc748 into volcengine:main Aug 29, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants