thejayman77
4110846c01
The Seed-3 fit of the Ch443f scanout hardening exposed one setup family in the EMIF (iopll_0_outclk0, 3.225 ns) domain: u_zc_emit|u_z|cache_dirty -> awaddr[*], WNS -0.062 / TNS -0.203, skew-dominated (-0.064 clock skew; the 5-level logic path itself would land ~+0.002 at zero skew). Not the Ch443f logic — the added EMIF-domain logic nudged this marginal Ch357-era control path negative. Fix (Codex-directed, minimal): the normal-eviction AW address is a pure function of the cache line's beat, known when the line is installed. Preload it in S_FILL_C alongside cache_beat<=pf_beat (awaddr <= ZBASE + (pf_beat<<5)), and drop the two later awaddr assignments gated by cache_dirty (the scene_flush branch and the dirty cache-miss branch) — those now only assert awvalid on the already-prepared address. Clear-path awaddr assignments unchanged. No new FSM state, pipeline stage, buffer, or protocol change. Safe: a freshly filled line cannot be dirty before S_FILL_C installs it; the address is stable across hits and scene flushes; after a dirty eviction the next fill re-runs S_FILL_C; clear invalidates the cache so the first subsequent fill overwrites the clear address before any normal eviction. This removes the control cone rather than placing around it, so closure is seed-robust. Regressions green: z_rmw/zc_emit/axi_master_elastic/zbuffer/z_flush_writer/ tile_zflush, full Ch443f scanout set, sh3_zint (Z 0/53760, COLOR 0/11092), sh3_zrop (errors=0), and f52 byte-identity (Z 0/307200, COLOR 0/245760, drops=0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
380 lines
23 KiB
Systemverilog
380 lines
23 KiB
Systemverilog
// retroDE_ps2 — gs_lpddr_z_rmw (Ch357 — packed PSMZ16S persistent-Z LPDDR read-modify-write engine, STANDALONE unit)
|
|
//
|
|
// Codex Ch357 gate 1: "Build the packed PSMZ16S LPDDR RMW engine as a standalone unit first." This is the depth-test
|
|
// core for Option B (LPDDR-persistent Z). It owns a PRIVATE, LINEAR, packed-16-bit Z buffer in LPDDR.
|
|
//
|
|
// AUTHENTICITY NOTE (Codex): this Z buffer is INTERNAL — it is NEVER exposed as GS local memory. Therefore only the
|
|
// PSMZ16S VALUE/TEST semantics are authentic (clamp16 source, GEQUAL, ZMSK); its PHYSICAL storage layout is deliberately a
|
|
// simple linear packing, NOT the PSMZ16S memory swizzle. That is valid precisely because nothing outside reads it as GS VRAM.
|
|
//
|
|
// PSMZ16S depth semantics (pinned against PCSX2 SW raster — GSRendererSW.cpp:1439 z_max=0xFFFF, GSDrawScanline…:1129
|
|
// source clamp, :1157 dest mask + GEQUAL):
|
|
// src_z = min(frag_z, 0xFFFF) // clamp16 (NOT mask)
|
|
// dest_z = stored 16-bit Z at the pixel
|
|
// pass = (src_z >= dest_z) // GEQUAL, larger Z = nearer
|
|
// on pass && !zmsk: stored := src_z // ZMSK=1 => test still runs, WRITE suppressed only
|
|
//
|
|
// Storage: pixel_index = y*FB_PXW + x ; 16 Z per 256-bit (32-byte) beat ; beat = pixel_index>>4, lane = pixel_index[3:0];
|
|
// byte addr = ZBASE + beat*32. Raster (scanline) order gives strong beat locality.
|
|
//
|
|
// Hazards / backpressure (Codex gates): a SINGLE write-back cache line (one 256-bit beat) holds all 16 lanes, so
|
|
// consecutive fragments to the same beat (incl. the SAME pixel) read the latest pending Z IN-PLACE — same-beat RMW hazards
|
|
// are forwarded with 1-cycle throughput and no stale read. On a beat MISS the engine flushes the dirty line, reads the new
|
|
// beat, and BACKPRESSURES the fragment producer (f_ready=0) throughout — a bounded producer, never an unbounded stream
|
|
// feeding a latent LPDDR read. Color is emitted by the CONSUMER only after p_pass (this unit produces the pass bit).
|
|
//
|
|
// Single clock (axi_clk domain). Integration adds the raster(gs_clk)->axi_clk CDC (async-FIFO, like gs_lpddr_axi_master).
|
|
// AXI4: single-beat INCR (arsize/awsize=5=32B, len=0), full per-byte wstrb, backpressured handshakes.
|
|
|
|
module gs_lpddr_z_rmw #(
|
|
parameter [31:0] ZBASE = 32'h0030_0000, // LPDDR byte base of the private Z buffer (Linux-safe reserved region)
|
|
parameter int FB_PXW = 256, // framebuffer pixel width (pixel-index stride)
|
|
parameter int FB_H = 210, // framebuffer height (bounds the preclear loop)
|
|
parameter [15:0] Z_CLEAR = 16'h0000 // GEQUAL clear value (0 = farthest; any fragment passes first)
|
|
) (
|
|
input logic clk,
|
|
input logic rst_n,
|
|
input logic enable, // 0 => fully inert (no AXI activity)
|
|
// ---- preclear (Codex: preclear shared Z once to the GEQUAL clear value) ----
|
|
input logic clear_start, // pulse: write Z_CLEAR to every beat, then clear_done
|
|
output logic clear_done,
|
|
// ---- scene-end flush (Codex: on the ordered end-of-scene marker, flush the dirty Z line + wait its BRESP). The cache
|
|
// stays VALID and CLEAN afterwards so Z PERSISTS across scheduler epochs (texture rebind must not clear it). ----
|
|
input logic scene_flush, // hold high until z_drained; flushes the dirty line to LPDDR
|
|
output logic z_drained, // all Z durable in LPDDR (idle, cache clean, no pending AXI)
|
|
// ---- fragment input stream (valid/ready) ----
|
|
input logic f_valid,
|
|
output logic f_ready,
|
|
input logic [11:0] f_x,
|
|
input logic [11:0] f_y,
|
|
input logic [31:0] f_z, // fragment Z (pre-clamp)
|
|
input logic f_zmsk, // 1 => suppress Z write (test still occurs)
|
|
input logic [1:0] f_ztst, // 0 NEVER, 1 ALWAYS, 2 GEQUAL, 3 GREATER
|
|
// ---- result output stream (in fragment order): p_pass gates the color write downstream ----
|
|
output logic p_valid,
|
|
input logic p_ready,
|
|
output logic p_pass,
|
|
output logic [11:0] p_x,
|
|
output logic [11:0] p_y,
|
|
output logic [15:0] p_zq, // the clamped fragment Z (for the color path / debug)
|
|
// ---- AXI4 read (Z fetch) ----
|
|
output logic [31:0] araddr,
|
|
output logic [7:0] arlen,
|
|
output logic [2:0] arsize,
|
|
output logic [1:0] arburst,
|
|
output logic arvalid,
|
|
input logic arready,
|
|
input logic [255:0] rdata,
|
|
input logic [1:0] rresp,
|
|
input logic rlast,
|
|
input logic rvalid,
|
|
output logic rready,
|
|
// ---- AXI4 write (Z flush) ----
|
|
output logic [31:0] awaddr,
|
|
output logic [7:0] awlen,
|
|
output logic [2:0] awsize,
|
|
output logic [1:0] awburst,
|
|
output logic awvalid,
|
|
input logic awready,
|
|
output logic [255:0] wdata,
|
|
output logic [31:0] wstrb,
|
|
output logic wlast,
|
|
output logic wvalid,
|
|
input logic wready,
|
|
input logic bvalid,
|
|
output logic bready,
|
|
input logic [1:0] bresp,
|
|
// ---- status ----
|
|
output logic [31:0] beats_read,
|
|
output logic [31:0] beats_written,
|
|
output logic [31:0] bresp_err,
|
|
output logic idle
|
|
);
|
|
localparam int NPX = FB_PXW*FB_H;
|
|
localparam int NBEATS = (NPX + 15) / 16;
|
|
localparam int BW = (NBEATS <= 1) ? 1 : $clog2(NBEATS);
|
|
|
|
// ---- clamp16 (PSMZ16S source clamp) ----
|
|
function automatic logic [15:0] clamp16(input logic [31:0] z);
|
|
clamp16 = (|z[31:16]) ? 16'hFFFF : z[15:0];
|
|
endfunction
|
|
|
|
// ---- single write-back Z-cache line ----
|
|
logic [255:0] cache_data; // 16 lanes x 16-bit Z
|
|
// Ch357 (Codex) — register the accepted AXI read data BEFORE updating cache_data, so the EMIF read-FIFO -> cache
|
|
// 310 MHz path is register->register (the -0.8 ns closure). S_FILL_R captures rdata here; S_FILL_C commits it.
|
|
logic [255:0] z_rd_q;
|
|
logic [BW-1:0] cache_beat;
|
|
logic cache_valid, cache_dirty;
|
|
|
|
// ---- pending fragment (latched during a miss) ----
|
|
logic [11:0] pf_x, pf_y;
|
|
logic [15:0] pf_zq;
|
|
logic pf_zmsk;
|
|
logic [1:0] pf_ztst;
|
|
logic [BW-1:0] pf_beat;
|
|
logic [3:0] pf_lane;
|
|
// Ch357 — DECODE-STAGE pipeline register (Codex: pipeline the 310 MHz u_req->cache path). The incoming fragment's
|
|
// index/lane/beat + clamp16(z) are computed combinationally and REGISTERED here on accept; the cache read/compare/
|
|
// write then runs the NEXT cycle off the registered fields. This splits the single-cycle FIFO->index->lane->256-bit
|
|
// cache-mux->compare->cache-write cone (routed -2.240 ns @ 310 MHz) into two shorter registered stages. Same-beat
|
|
// forwarding is preserved: fragments are 2 cycles apart in the RMW stage, so an in-place cache write is committed a
|
|
// cycle before the next same-beat read. Throughput 155 MHz >> the raster fragment rate (no new FIFO pressure).
|
|
logic d_valid;
|
|
logic [3:0] d_lane;
|
|
logic [15:0] d_zq;
|
|
logic d_zmsk;
|
|
logic [1:0] d_ztst;
|
|
logic [11:0] d_x, d_y;
|
|
logic [BW-1:0] d_beat;
|
|
// Ch358 (Codex) — registered HIT flag, the companion to d_beat: loaded from hit_c at stage-1 accept so stage 2
|
|
// branches on ONE registered bit instead of the BW-wide (15-bit at 640x480) d_beat==cache_beat equality that
|
|
// directly gated the 256-bit wdata<=cache_data dirty-evict load (the Ch358 fit's WNS -0.092 family). Exact by
|
|
// construction: accept happens only in S_RUN with the decode slot free (f_ready), and cache_beat/cache_valid
|
|
// change only in S_FILL_C (unreachable while d_* is pending) or at clear_start (precedes all fragments; the
|
|
// zint gates assert pre_clear_frags==0) -> hit_c cannot go stale between accept and the single stage-2 consume
|
|
// (a missed fragment is promoted to pf_* and handed to rmw_* by S_FILL_C; it never re-enters stage 2).
|
|
logic d_hit;
|
|
// Ch357 (Codex) — RMW-STAGE pipeline register (stage 3). Stage 2 does the barrel READ of the target lane
|
|
// (cache_data[d_lane] on a hit, z_rd_q[pf_lane] on a fill) into rmw_dz; stage 3 (next cycle) does GEQUAL + the
|
|
// barrel WRITE via an explicit 16-way case. Splitting read from compare+write removes the cache_data->cache_data
|
|
// cross-lane feedback mux (the -0.607ns 310MHz path). f_ready already spaces fragments 2 cyc apart, so an in-place
|
|
// lane write commits one cycle before the next same-beat lane read -> same-beat forwarding preserved, no bypass.
|
|
logic rmw_valid;
|
|
logic [3:0] rmw_lane;
|
|
logic [15:0] rmw_dz; // the read-out dest Z of the target lane (registered)
|
|
logic [15:0] rmw_zq; // clamped fragment Z
|
|
logic rmw_zmsk;
|
|
logic [1:0] rmw_ztst;
|
|
logic [11:0] rmw_x, rmw_y;
|
|
// Ch367 — register GEQUAL before it enables a cache-line update. This
|
|
// removes the compare -> 16-way cache_data write-enable cone at 310 MHz.
|
|
logic cmp_valid;
|
|
logic cmp_pass, cmp_write;
|
|
logic [3:0] cmp_lane;
|
|
logic [15:0] cmp_zq;
|
|
logic [11:0] cmp_x, cmp_y;
|
|
|
|
// combinational address/lane of the INCOMING fragment
|
|
logic [31:0] px_index_c;
|
|
logic [BW-1:0] beat_c;
|
|
logic [3:0] lane_c;
|
|
always_comb begin
|
|
px_index_c = f_y*FB_PXW + f_x;
|
|
beat_c = px_index_c[4 +: BW];
|
|
lane_c = px_index_c[3:0];
|
|
end
|
|
wire hit_c = cache_valid && (beat_c == cache_beat);
|
|
|
|
// ---- result register (1-deep skid; holds until p_ready) ----
|
|
logic res_full;
|
|
logic res_pass;
|
|
logic [11:0] res_x, res_y;
|
|
logic [15:0] res_zq;
|
|
assign p_valid = res_full;
|
|
assign p_pass = res_pass;
|
|
assign p_x = res_x;
|
|
assign p_y = res_y;
|
|
assign p_zq = res_zq;
|
|
|
|
// ---- FSM ----
|
|
typedef enum logic [3:0] { S_RUN, S_FLUSH_AW, S_FLUSH_B, S_FILL_AR, S_FILL_R, S_FILL_C, S_CLR, S_CLR_B,
|
|
S_SFLUSH_AW, S_SFLUSH_B } st_t;
|
|
st_t st;
|
|
logic [BW-1:0] clr_beat;
|
|
|
|
// helper: perform the RMW on the cache line for a given lane/zq/zmsk, return pass (comb) and next cache line
|
|
function automatic logic ztest_pass(input logic [1:0] op, input logic [15:0] s, input logic [15:0] d);
|
|
case (op)
|
|
2'd0: ztest_pass = 1'b0;
|
|
2'd1: ztest_pass = 1'b1;
|
|
2'd2: ztest_pass = (s >= d);
|
|
2'd3: ztest_pass = (s > d);
|
|
endcase
|
|
endfunction
|
|
|
|
// Ch357 (Codex) — EXPLICIT 16-way lane write (constant slices, NOT dynamic cache_data[lane*16+:16]). Each arm writes
|
|
// one fixed 16-bit slice; the other 15 lanes are unassigned -> HOLD. This is a per-lane write-enable, not a 256-bit
|
|
// cross-lane feedback mux, which is what kept the compare+write out of the critical cone.
|
|
task automatic cache_write_lane(input logic [3:0] lane, input logic [15:0] val);
|
|
case (lane)
|
|
4'd0: cache_data[ 15: 0] <= val; 4'd1: cache_data[ 31: 16] <= val;
|
|
4'd2: cache_data[ 47: 32] <= val; 4'd3: cache_data[ 63: 48] <= val;
|
|
4'd4: cache_data[ 79: 64] <= val; 4'd5: cache_data[ 95: 80] <= val;
|
|
4'd6: cache_data[111: 96] <= val; 4'd7: cache_data[127:112] <= val;
|
|
4'd8: cache_data[143:128] <= val; 4'd9: cache_data[159:144] <= val;
|
|
4'd10: cache_data[175:160] <= val; 4'd11: cache_data[191:176] <= val;
|
|
4'd12: cache_data[207:192] <= val; 4'd13: cache_data[223:208] <= val;
|
|
4'd14: cache_data[239:224] <= val; 4'd15: cache_data[255:240] <= val;
|
|
endcase
|
|
endtask
|
|
|
|
always_ff @(posedge clk or negedge rst_n) begin
|
|
if (!rst_n) begin
|
|
st<=S_RUN; cache_valid<=1'b0; cache_dirty<=1'b0; cache_beat<='0; cache_data<='0; z_rd_q<='0;
|
|
res_full<=1'b0; res_pass<=1'b0; res_x<='0; res_y<='0; res_zq<='0;
|
|
pf_x<='0; pf_y<='0; pf_zq<='0; pf_zmsk<='0; pf_ztst<=2'd2; pf_beat<='0; pf_lane<='0;
|
|
d_valid<=1'b0; d_lane<='0; d_zq<='0; d_zmsk<='0; d_ztst<=2'd2; d_x<='0; d_y<='0; d_beat<='0; d_hit<=1'b0;
|
|
rmw_valid<=1'b0; rmw_lane<='0; rmw_dz<='0; rmw_zq<='0; rmw_zmsk<='0; rmw_ztst<=2'd2; rmw_x<='0; rmw_y<='0;
|
|
cmp_valid<=1'b0; cmp_pass<=1'b0; cmp_write<=1'b0; cmp_lane<='0; cmp_zq<='0; cmp_x<='0; cmp_y<='0;
|
|
arvalid<=1'b0; araddr<='0; rready<=1'b0;
|
|
awvalid<=1'b0; awaddr<='0; wvalid<=1'b0; wdata<='0; wlast<=1'b0; bready<=1'b0;
|
|
beats_read<='0; beats_written<='0; bresp_err<='0; clr_beat<='0; clear_done<=1'b0;
|
|
end else begin
|
|
// constant AXI framing
|
|
arlen<=8'd0; arsize<=3'd5; arburst<=2'b01;
|
|
awlen<=8'd0; awsize<=3'd5; awburst<=2'b01; wstrb<=32'hFFFF_FFFF;
|
|
|
|
// clear the result reg when the consumer takes it
|
|
if (res_full && p_ready) res_full<=1'b0;
|
|
|
|
if (clear_start && st==S_RUN) begin
|
|
cache_valid<=1'b0; cache_dirty<=1'b0; clear_done<=1'b0; clr_beat<='0;
|
|
awaddr<=ZBASE; awvalid<=1'b1; wdata<={16{Z_CLEAR}}; wvalid<=1'b1; wlast<=1'b1; st<=S_CLR;
|
|
end
|
|
|
|
case (st)
|
|
// ---------------- normal processing ----------------
|
|
S_RUN: begin
|
|
// STAGE 4 — write a registered decision and publish its result. This is the only normal
|
|
// cache_data writer, and its enable is now register-local.
|
|
if (cmp_valid && (!res_full || p_ready)) begin
|
|
if (cmp_write) begin cache_write_lane(cmp_lane, cmp_zq); cache_dirty<=1'b1; end
|
|
res_full<=1'b1; res_pass<=cmp_pass; res_x<=cmp_x; res_y<=cmp_y; res_zq<=cmp_zq;
|
|
cmp_valid<=1'b0;
|
|
end
|
|
|
|
// STAGE 3 — register GEQUAL + write intent. Stage 2 waits for cmp_valid to clear, so a
|
|
// same-beat successor never samples cache_data in the cycle its predecessor writes it.
|
|
if (rmw_valid && !cmp_valid) begin
|
|
cmp_valid<=1'b1; cmp_pass<=ztest_pass(rmw_ztst, rmw_zq, rmw_dz);
|
|
cmp_write<=ztest_pass(rmw_ztst, rmw_zq, rmw_dz) && !rmw_zmsk;
|
|
cmp_lane<=rmw_lane; cmp_zq<=rmw_zq; cmp_x<=rmw_x; cmp_y<=rmw_y;
|
|
rmw_valid<=1'b0;
|
|
end
|
|
|
|
// scene-end flush (ordered after all fragments): push the dirty line, keep it cached (persist across
|
|
// epochs). Waits for every pipeline stage to drain so nothing is stranded.
|
|
if (scene_flush && !d_valid && !rmw_valid && !cmp_valid && cache_valid && cache_dirty) begin
|
|
awvalid<=1'b1; // Ch443f — awaddr already prepared at S_FILL_C install
|
|
wdata<=cache_data; wvalid<=1'b1; wlast<=1'b1; st<=S_SFLUSH_AW;
|
|
end
|
|
// STAGE 2 — barrel READ of the target lane into the RMW register (hit), or promote+fill (miss). Fires
|
|
// only when the RMW slot is empty (!rmw_valid): with f_ready's 2-cyc spacing the stages alternate, so
|
|
// stage 2 never collides with a same-cycle stage-3 cache write (miss flush reads a settled cache_data).
|
|
else if (d_valid && !rmw_valid && !cmp_valid) begin
|
|
d_valid <= 1'b0;
|
|
if (d_hit) begin // Ch358 — registered at accept (== cache_valid && d_beat==cache_beat there)
|
|
// HIT: register the read-out dest Z; stage 3 compares + writes next cycle.
|
|
rmw_valid<=1'b1; rmw_dz<=cache_data[d_lane*16 +: 16];
|
|
rmw_lane<=d_lane; rmw_zq<=d_zq; rmw_zmsk<=d_zmsk; rmw_ztst<=d_ztst; rmw_x<=d_x; rmw_y<=d_y;
|
|
end else begin
|
|
// MISS: promote the decoded fragment to pf_*, backpressure, flush-if-dirty then fill
|
|
pf_x<=d_x; pf_y<=d_y; pf_zq<=d_zq; pf_zmsk<=d_zmsk; pf_ztst<=d_ztst; pf_beat<=d_beat; pf_lane<=d_lane;
|
|
if (cache_valid && cache_dirty) begin
|
|
awvalid<=1'b1; // Ch443f — awaddr already prepared at S_FILL_C install
|
|
wdata<=cache_data; wvalid<=1'b1; wlast<=1'b1; st<=S_FLUSH_AW;
|
|
end else begin
|
|
araddr<=ZBASE + (d_beat<<5); arvalid<=1'b1; st<=S_FILL_AR;
|
|
end
|
|
end
|
|
end
|
|
// ACCEPT (stage-1): register the combinational index/lane/beat + clamp16(z). f_ready gates this to
|
|
// one decoded fragment in flight (d_valid), so accept and stage 2 never collide on d_* (2 cyc/frag).
|
|
if (enable && f_valid && f_ready) begin
|
|
d_valid<=1'b1; d_lane<=lane_c; d_zq<=clamp16(f_z); d_zmsk<=f_zmsk; d_ztst<=f_ztst;
|
|
d_x<=f_x; d_y<=f_y; d_beat<=beat_c; d_hit<=hit_c; // Ch358 — pre-registered hit decision
|
|
end
|
|
end
|
|
// ---------------- flush the dirty line (AW then W then B) ----------------
|
|
S_FLUSH_AW: begin
|
|
if (awready) awvalid<=1'b0;
|
|
if (wready) wvalid <=1'b0;
|
|
if ((awready||!awvalid) && (wready||!wvalid)) begin bready<=1'b1; st<=S_FLUSH_B; end
|
|
end
|
|
S_FLUSH_B: begin
|
|
if (bvalid) begin
|
|
bready<=1'b0; beats_written<=beats_written+1;
|
|
if (bresp!=2'b00) bresp_err<=bresp_err+1;
|
|
cache_dirty<=1'b0;
|
|
araddr<=ZBASE + (pf_beat<<5); arvalid<=1'b1; st<=S_FILL_AR; // now fill the wanted beat
|
|
end
|
|
end
|
|
// ---------------- fill the wanted beat (AR then R) ----------------
|
|
S_FILL_AR: begin
|
|
if (arready) begin arvalid<=1'b0; rready<=1'b1; st<=S_FILL_R; end
|
|
end
|
|
S_FILL_R: begin
|
|
// Ch357 — ACCEPT: register the read beat (z_rd_q) and take the R handshake; the EMIF read-FIFO ->
|
|
// z_rd_q path is now register->register. The cache update happens next cycle in S_FILL_C.
|
|
if (rvalid) begin
|
|
rready<=1'b0; beats_read<=beats_read+1;
|
|
if (rresp!=2'b00) bresp_err<=bresp_err+1;
|
|
z_rd_q<=rdata; st<=S_FILL_C;
|
|
end
|
|
end
|
|
S_FILL_C: begin
|
|
// Ch357 — COMMIT: fill the cache line from the registered beat, and hand the pending fragment to the
|
|
// RMW stage (barrel READ of the target lane off the just-read beat). Stage 3 does the GEQUAL + 16-way
|
|
// lane write next cycle in S_RUN (a guaranteed hit). Same result as the old single-cycle fill, staged.
|
|
cache_data<=z_rd_q; cache_beat<=pf_beat; cache_valid<=1'b1; cache_dirty<=1'b0;
|
|
// Ch443f — PRELOAD the line's normal-eviction AW address at install. The address
|
|
// is a pure function of this line's beat (== cache_beat after this cycle) and is
|
|
// stable across every hit/flush until the next fill re-installs it. Preparing it
|
|
// here removes the 5-level cache_dirty -> awaddr control cone that was the EMIF
|
|
// setup leader; the eviction branches below now only assert awvalid.
|
|
awaddr<=ZBASE + (pf_beat<<5);
|
|
rmw_valid<=1'b1; rmw_dz<=z_rd_q[pf_lane*16 +: 16];
|
|
rmw_lane<=pf_lane; rmw_zq<=pf_zq; rmw_zmsk<=pf_zmsk; rmw_ztst<=pf_ztst; rmw_x<=pf_x; rmw_y<=pf_y;
|
|
st<=S_RUN;
|
|
end
|
|
// ---------------- preclear: write Z_CLEAR to every beat (issue -> handshake -> B, loop) ----------------
|
|
S_CLR: begin
|
|
if (awready) awvalid<=1'b0;
|
|
if (wready) wvalid <=1'b0;
|
|
if ((awready||!awvalid) && (wready||!wvalid)) begin bready<=1'b1; st<=S_CLR_B; end
|
|
end
|
|
S_CLR_B: begin
|
|
if (bvalid && bready) begin
|
|
bready<=1'b0; beats_written<=beats_written+1;
|
|
if (bresp!=2'b00) bresp_err<=bresp_err+1;
|
|
if (clr_beat==BW'(NBEATS-1)) begin clear_done<=1'b1; st<=S_RUN; end
|
|
else begin
|
|
clr_beat<=clr_beat+1'b1;
|
|
awaddr<=ZBASE + ((clr_beat+1'b1)<<5); awvalid<=1'b1; wvalid<=1'b1; wlast<=1'b1; st<=S_CLR;
|
|
end
|
|
end
|
|
end
|
|
// ---------------- scene-end flush: write the dirty line, keep cache VALID+CLEAN (Z persists) ----------------
|
|
S_SFLUSH_AW: begin
|
|
if (awready) awvalid<=1'b0;
|
|
if (wready) wvalid <=1'b0;
|
|
if ((awready||!awvalid) && (wready||!wvalid)) begin bready<=1'b1; st<=S_SFLUSH_B; end
|
|
end
|
|
S_SFLUSH_B: begin
|
|
if (bvalid) begin
|
|
bready<=1'b0; beats_written<=beats_written+1;
|
|
if (bresp!=2'b00) bresp_err<=bresp_err+1;
|
|
cache_dirty<=1'b0; st<=S_RUN; // cache remains VALID (persist across epochs), now clean/durable
|
|
end
|
|
end
|
|
default: st<=S_RUN;
|
|
endcase
|
|
end
|
|
end
|
|
|
|
// f_ready: accept a new fragment in S_RUN (hit => processed this cycle; miss => latched, then flush/fill), whenever the
|
|
// result slot is free or draining this cycle. On a miss the drain empties res_full and it stays empty until S_FILL_R
|
|
// re-fills it with the pending fragment's result, so no clobber.
|
|
// hold off new fragments during a scene flush too (so the marker's flush isn't interleaved with a fresh RMW)
|
|
// Ch367 pipeline — accept only when every local stage is clear. The extra compare-result stage preserves
|
|
// same-beat forwarding without a bypass: the prior write commits before the successor can read cache_data.
|
|
assign f_ready = enable && (st==S_RUN) && !(scene_flush && cache_valid && cache_dirty)
|
|
&& !d_valid && !rmw_valid && !cmp_valid;
|
|
// idle / z_drained must see all three stages drained, else a fragment would be lost at a scene boundary.
|
|
assign idle = (st==S_RUN) && !d_valid && !rmw_valid && !cmp_valid && !res_full && !arvalid && !awvalid && !wvalid && !bready;
|
|
// z_drained: all Z durable in LPDDR — no local work, no dirty line, no pending AXI, no result stuck.
|
|
assign z_drained = (st==S_RUN) && !d_valid && !rmw_valid && !cmp_valid && !cache_dirty && !res_full && !arvalid && !awvalid && !wvalid && !bready;
|
|
|
|
endmodule : gs_lpddr_z_rmw
|